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CHAPTER ONE 


Overview 





SPECIAL REPORT 2000 


Under section 12 of the Audit Act, the Provincial Auditor is required to report annually to the 
Speaker of the Legislative Assembly after the fiscal year end is closed and the Public Accounts 
have been laid before the Assembly. This year, there was a delay in the Public Accounts being 
laid before the Assembly. In fact, by October 13, 2000, my Office had not received a final draft 
of the province’s Financial Statements from the Ministry of Finance to enable us to finalize the 
Auditor’s Report on the Public Accounts. As a result, we could also not finalize the parts of our 
Annual Report that relate to the Public Accounts. However, other, significant portions of our 
report had been completed and were ready for publication; specifically, our value for money 
audit reports and follow-ups of recommendations made in our 1998 Annual Report. In my 
opinion, these matters should not be deferred until the Annual Report is tabled. Therefore, I am 
tabling this Special Report on Accountability and Value for Money in order that the 

Legislature and the Standing Committee on Public Accounts be informed of the results of these 
audit activities and be able to take action in a timely manner. The Annual Report, containing my 
opinion and comments on the financial statements of the province, will be tabled after the 
Public Accounts have been tabled. 


ACCOUNTABILITY 


Chapter Two addresses specific issues of governance and accountability in government. This 
year there are two issues that I believe warrant discussion to improve accountability to the 
Legislature for the prudent use of public funds: concerns regarding public accountability for the 
Ontario Innovation Trust and legislative proposals for increased public accountability. 


With respect to the $750 million flowed to the Ontario Innovation Trust, I believe that 
accountability to the Legislature, and therefore to the taxpayers, for the expenditure of these 
public funds is significantly impaired. As well, $500 million of this amount was preflowed to the 
Trust well before the Trust actually required the funds. 


About half of the province’s annual expenditures, approximately $30 billion, are spent by the 
government as transfer payments to government service delivery agents. The Audit Act 
currently does not permit my Office to access on a discretionary basis all the information 
necessary to report to the Legislature the extent to which these agents achieve intended results 
and whether or not taxpayers are receiving value for money spent. Accordingly, we have 
sought over the last decade to have the Audit Act amended to enable my Office to assist the 
Legislature in strengthening public accountability. 


Over four years ago, the Standing Committee on Public Accounts unanimously endorsed our 
proposed amendments to the Audit Act, but there has been no action in this area on the part of the 
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government. In my opinion, the proposed revisions deserve more serious and timely 
consideration as they will enable my Office to better and more comprehensively serve the 
Legislature and thereby the taxpayers of Ontario. I recommend that the Committee revisit this 
subject in its upcoming sitting. 


With regard to the proposed Public Sector Accountability Act, I continue to be a strong 
advocate of any legislation that enhances public sector accountability. 


VALUE FOR MONEY AUDIT 
RECOMMENDATIONS 


OVERALL COMMENTS 


As I have emphasized in previous reports, having good information for decision making is 
essential. Appropriate, reliable and timely information enables decision makers to accurately 
assess the economy, efficiency and effectiveness of government programs and activities. Such 
information provides a critical base for decision makers to decide whether to continue, 
discontinue or change government programs and activities, including the use of alternative 
service delivery or common purpose procurement. Good administration of public funds depends 
on good decisions based on good information. 


This year, one of the significant themes of my Special Report, as in past reports, is that 
ministries still require improvements in the quality of their information about the economy and 
efficiency of the programs and services they deliver. We found that they often lacked adequate 
procedures for measuring and reporting on program effectiveness. There are significant 
opportunities to improve information for decision making, which would lead to an improved 
administration of public funds and better performance in delivering government services. 


I would like to highlight some of the areas where we identified the need for significant 
improvements: 


* Over 50% of land ambulance operators were not meeting established response time 
requirements, which were based on 1996 actual response times. 


The land ambulance system, when realigned through downloading to municipalities, may not 
provide a balanced and integrated system of services and may cost Ontarians an additional 
$100 million to achieve 1996 actual response times. 


*  Agricorp failed to manage certain of its resources with due regard for economy and 
efficiency and, on a number of occasions, failed to safeguard the resources entrusted to it. 


* The Ministry of the Environment did not know the extent to which facilities that discharge 
contaminants into the environment were meeting current environmental standards and 
consequently, where corrective action had to be taken. 


A 25% reduction in staff at the Ministry over the last few years had contributed to a 34% 
decrease in the number of ministry-initiated inspections conducted per year. 


In the process of implementing an infrastructure renewal project with estimated capital 
costs of over $270 million, the Ministry of Correctional Services did not properly 
assess the viability of alternative delivery options to ensure best value to the taxpayers. 
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The estimated cost to build a new cooking facility to serve a number of correctional 
institutions had increased from $5 million to $9.5 million. As well, the facility’s production 
capacity would be 1,000 meals a day short of meeting the needs of the institutions to be 
served. 


¢ The project to automate the land registration system (POLARIS) was transferred in 1991 
by the Ministry of Consumer and Commercial Relations to Teranet Land Information 
Services Inc. (Teranet). According to a consultant’s study, the 1991 cost estimate of 
$275 million to complete POLARIS could now be as high as $1 billion, and though the 
original anticipated completion date was 1999, Teranet has indicated a project completion 
date of 2010. 


THE AUDITING AND REPORTING PROCESS 


Because of the size and complexity of the province’s operations and administration, it is 
impossible to audit each program every year. Instead, the Office selects the audits it conducts 
in a cycle, so that all major programs are considered for coverage every five years. The audits 
covered by this Special Report were selected by the Office’s senior management based on 
criteria such as financial impact, significance to the Legislative Assembly, public sensitivity and 
safety, and past audit reports. 


We plan, perform and report our value for money work in accordance with the professional 
standards for assurance engagements, encompassing value for money and compliance, 
established by the Canadian Institute of Chartered Accountants. 


Before beginning an audit, staff meet with auditee representatives to discuss the focus of the 
audit in general terms. During the audit, staff maintain an ongoing dialogue with the auditee to 
review the progress of the audit and ensure open lines of communication. After the audit, staff 
conclude their on-site work, then a draft report is prepared, reviewed internally and discussed 
with the auditee. A management response to our recommendations is incorporated into the final 
draft report. The Provincial Auditor and senior office staff meet with the deputy minister or 
agency head to discuss the final draft report and to finalize the responses. Those responses are 
provided in the report sections that comprise Chapter Three of this Special Report. 


Immediately prior to the tabling of our reports to the Legislative Assembly, separate and 
simultaneous lockups are arranged for members of the Legislative Assembly and their research 
staff, representatives of the media, and representatives of audited ministries and agencies. 
When the lockups conclude, the Provincial Auditor is available to answer questions from media 
representatives. 


Each year, the Standing Committee on Public Accounts selects sections of the Provincial 
Auditor’s report for review and calls upon representatives of the audited ministries and 
agencies to attend as witnesses. 


Since 1993 it has been our practice to make specific recommendations in our value for money 
audits and reviews for corrective action by ministries and agencies and, two years after the 
publication of the recommendations in our report, to follow up on the status of actions taken. 
Chapter Four of this report contains our comments on the current status of actions taken on the 
recommendations made in our 1998 Annual Report. 
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VALUE FOR MONEY REPORT SUMMARIES 


The following are summaries of the 15 reports on value for money audits and reviews 
contained in Chapter Three of this Special Report. The auditees’ responses in Chapter Three 
indicate that action to implement many of our recommendations has been planned or has 
already been taken. 


3.01 Ministry of Agriculture, Food and Rural Affairs 
AgriCorp 


Established by the AgriCorp Act, 1996, AgriCorp’s primary business is to administer insurance 
plans for the agriculture and food industry. As of March 31, 2000, AgriCorp held assets totalling 
$604 million in its General Fund, Crop Insurance Fund and Market Revenue Program fund. 
AgriCorp’s General Fund paid administrative expenditures totalling over $16 million for the 
fiscal year ended March 31, 2000. During the same fiscal year, the other two funds paid $180 
million to compensate Ontario farmers for losses from reduced crop yields and low market 
prices. 


We concluded that AgriCorp did not have adequate procedures in place to ensure that its 
activities complied with legislation and corporate procedures. In addition, AgriCorp failed to 
manage certain of its resources with due regard for economy and efficiency and, on a number 
of occasions, failed to safeguard the resources entrusted to it. Also, AgriCorp did not have the 
necessary governance and accountability procedures in place to ensure that the Corporation 
was well managed or to provide the information required to measure and report on its 
effectiveness. 


Some of our specific findings were: 


*  Inappropriately utilizing ministry funds, AgriCorp lost $325,000 in a speculative investment 
initiative to buy and sell bonds on a daily basis. AgriCorp also violated its fiduciary 
responsibility by attempting to transfer the loss to the Ontario Crop Insurance Fund. My 
Office had to intervene to reverse this transfer. 


* Contrary to legislation, on several occasions AgriCorp sought to remove money from the 
Ontario Crop Insurance Fund to pay for its administrative expenses. My Office had to 
intervene to ensure the Fund remained intact. 


* Without a proper business case and without tender, AgriCorp engaged an intermediary to 
place $14.5 million of reinsurance coverage with private insurance companies. 


* AgriCorp received little or no value for the $3 million it spent on information technology 
development projects that were poorly planned, controlled and managed. In addition, 
information technology consultants were engaged without competition and provided with 
training at taxpayers’ expense, and several consultants were retained for periods ranging 
from five to 13 years at rates of up to $640 per day. 


* AgriCorp engaged an investment advisor for a minimum annual fee of $400,000 without 
competition. The advice received was of little value as the advisor repeatedly 
recommended investments to AgriCorp that were contrary to its legislation. 


Accordingly, we made a number of recommendations to improve the management and 


governance of AgriCorp and received commitments from AgriCorp and the Ministry that they 
would take corrective action. 
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3.02 Ministry of Community and Social Services 
Child Welfare Services Program 


The Ministry is responsible for setting expectations for child welfare services through legislation 
and regulations, as well as for funding and monitoring services provided by Children’s Aid 
Societies (Societies) and taking corrective action where necessary. Services provided by the 
Societies include: investigating and assessing child abuse and neglect allegations, as well as 
providing counselling, guardianship and many other types of services that protect children from 
neglect and physical, sexual and emotional abuse. 


The Ministry estimated that 154,000 children from 86,000 families received some type of 
service from a Society during the fiscal year ending March 31, 2000. Program expenditures 
during that year totalled approximately $654 million, of which $642 million was for transfer 
payments to the Societies. 


Our principal conclusion was that the Ministry did not have sufficient assurance that children in 
need were adequately protected because: 


¢ Societies could not always demonstrate that they conducted their assessments of children 
reported to be in need; 


¢ not all plans of service, which outline the actions that need to be taken to protect a child, 
were prepared or implemented on a timely basis by the Societies; and 


* program outcome measures had not been developed and implemented. 


We also concluded that the Ministry’s new funding framework, which is to be fully implemented 
in the 2000/01 fiscal year, is a substantial improvement over the Ministry’s previous method of 
determining the Societies’ program funding. However, the Ministry will not realize the full 
benefits of the new funding framework until direct and indirect service costs to be funded are 
linked to the nature and assessed cost of the underlying services to be received. 


We made recommendations to overcome these deficiencies and the Ministry responded to our 
recommendations with commitments to take corrective action. 


3.03 Ministry of Consumer and Commercial Relations 


Project to Automate the Land Registration System (POLARIS) 


In 1980, the Ministry established POLARIS (Province of Ontario Land Registration System), a 
project to automate the province’s land registration system. The project involves the conversion 
and automation of paper-based records to permit the searching and registration through 
computers of real property documents. In 1991, the Ministry transferred ownership and 
responsibility for the implementation and operation of POLARIS to Teranet Land Information 
Services Inc. (Teranet), a corporation owned jointly by the province, with 40% of the voting 
shares, and a private sector company, with 60% of the voting shares. As of March 31, 2000, 
approximately 2.5 million of the estimated 4.3 million properties in the province had been fully 
converted to electronic format. 

Teranet receives fees collected by the Ministry for registration and ancillary transactions that 
the Ministry processes using POLARIS. In return, the Ministry receives royalties from Teranet 


for registration-related revenue and for ancillary and certain other services. Since 1991, the 
Ministry has, out of the Consolidated Revenue Fund, paid Teranet $235 million from revenue 
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obtained for automated land registration and ancillary services, of which Teranet has paid back 
$45 million in royalties to the Ministry. 


During our review, we identified a number of significant concerns with the Ministry’s 
arrangement with Teranet and the status of its project to automate the province’s land 
registration system, including: 


* In 1991, the POLARIS project had an anticipated 1999 completion date, but at the time of 
our review, Ministry information received from Teranet indicated a 2010 project completion 
date. 


* Cost estimates to complete the project had substantially increased from the original 1991 
estimate of $275 million. As of April 1999, Teranet estimated that total costs for the project 
would be more than $700 million—an estimate based on Teranet’s assumption that certain 
cost-saving methods could be implemented. A consultant engaged by the Ministry to 
provide advice on the Ministry’s financial risk with respect to the project noted that other, 
less favourable scenarios estimate the total project costs to exceed $1 billion. 


* The consultant also noted that the cost to convert the remaining 1.8 million properties using 
existing workflows and processes exceeds the anticipated revenues from conversions. 


* There is a risk that if the Ministry were required to terminate its agreement with Teranet 
and assume operation and control of the POLARIS project, it would have to address 
Teranet’s $280-million obligation to its bondholders. As well, the Ministry may have to 
compensate Teranet for all or part of the $300 million in costs Teranet has incurred. 


The Ministry advised us that it had not agreed to changes to the contractual obligations of 
Teranet for completing the project and that it was in the process of deciding on an appropriate 
course of action. We plan to follow up on the Ministry’s progress in due course. 


3.04 Ministry of Correctional Services 
Institutional Services and Young Offender Operations 


The Ministry’s Institutional Services and Young Offender Operations (Institutional Services) is 
responsible for the operation of Ontario’s correctional institutions. These institutions provide 
custody for adult offenders sentenced to terms of up to two years less a day and for accused 
persons on remand awaiting trial. They also provide custody for young offenders between 16 
and 17 years of age. 


For 1999/2000, Institutional Services had operating expenditures of approximately $463 million 
and about 6,200 staff. On a daily basis there were approximately 7,400 adult and 700 young 
offenders in 47 correctional institutions. 


We concluded that in the process of implementing an infrastructure renewal project with 
estimated capital costs of over $270 million, the Ministry did not properly assess the viability 
of alternative delivery options to ensure best value to taxpayers. In that regard, we found that: 


* The Ministry’s decision to finance and build two 1,200-bed correctional institutions at 
a cost of $180 million was not supported by a comprehensive business case assessing 
the risks, costs and benefits of all relevant alternatives. 


* A proper business case and cost-benefit analysis were not done for building a new cooking 
facility within a correctional institution under expansion to provide prepared food to a 
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number of correctional institutions. At the completion of our fieldwork, the estimated cost to 
build the facility had increased from $5 million to $9.5 million. As well, the facility’s 
production capacity would be 1,000 meals a day short of meeting the needs of the 
correctional institutions to be served. 


In addition, despite a decline in average inmate count in recent years, the operating 
expenditures for Institutional Services between 1995/96 and 1999/2000 had increased 19% 
from $388 million to $463 million. Specifically, we noted that: 


¢ The number of offenders in the Temporary Absence Program, which is designed to protect 
society by enabling non-violent offenders to maintain community and family relationships 
and responsibilities, had declined from 25,000 to 4,000 between 1991/92 and 1998/99. The 
under-utilization of the community program resulted in the Ministry foregoing significant 
potential savings of as much as $50 million a year. 


¢ The average number of sick days per correctional officer increased 38% from 12 days in 
1995 to 16 days in 1998. Overtime expenditures increased 48% from $11.1 million in 
1996/97 to $16.5 million in 1998/99. 


We also found that about 60% of the Ministry’s 47 correctional institutions had security non- 
compliance problems that had not been rectified for up to two years. 


We made a number of recommendations for improvement and received commitments from the 
Ministry that it would take corrective action. 


3.05 Ministry of Education 
Pupil Transportation Grants to School Boards 


The transportation of students from home to school and back is a major undertaking as 


approximately 800,000 students are eligible for service. The Ministry of Education’s 
transportation grants to school boards for the 1999/2000 school year totalled $575 million. 


These grants were based on the 1997 transportation expenditures of school boards with 
adjustments for changes in enrolment. The Ministry was in the process of designing a grant 
formula that is based on need rather than enrolment and expected to implement the new 
formula in time for the school boards’ 2001/02 fiscal year. 


We concluded that the Ministry, which was working toward its five-year plan to revise its 
funding and accountability relationship with school boards, still had to establish satisfactory 
systems and procedures to ensure that: 


* school boards and the Ministry have the information needed to measure and report on 
transportation service performance; 


° the Ministry has the information needed to establish equitable funding for these services; 
and 


- the costs to provide transportation services to high needs students are properly tracked and 
managed. 


We made a number of recommendations for improvement and received commitments from 
the Ministry that it would take corrective action. 
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We also assessed whether the Ministry of Transportation had established satisfactory systems 
and procedures to fulfil its statutory responsibilities with respect to operators of school purpose 
vehicles and to contribute to the safe transportation of students (see Section 3.15). 


3.06 Ministry of the Environment 
Operations Division 


The Operations Division is responsible for administering the Ministry’s approvals and 
enforcement activities. It also responds to reports of pollution and spills that may have health 
and environmental impacts. In addition, the Division cleans up abandoned contaminated sites 
using funds available in the Environmental Clean-Up Fund. 


For the 1999/2000 fiscal year, the Division had total expenditures of $62 million and about 680 
staff. Since 1994, the Division had reduced its staff level by over 25%. 


We concluded that the Ministry did not have satisfactory systems and procedures in place to 
administer approvals and to enforce compliance with environmental legislation. Our major 
concerns included: 


* The Ministry’s systems did not enable it to assess whether and to what extent the over 
220,000 certificates of approval issued since 1957 were up to date. Certificates of approval 
are required for any facility that discharges contaminants into the environment. As a result, 
the Ministry did not know the extent to which facilities were not meeting current 
environmental standards and, consequently, where corrective action had to be taken. 


* Over $90 million in financial assurance was not obtained from facility operators, as required 
under legislation, to clean up potential damages that operators may cause to the 
environment. 


* A 25% reduction in staff over the last four years had contributed to a 34% decrease in the 
number of ministry-initiated inspections conducted per year. Further, the Ministry relied 
extensively on facility operators to comply voluntarily rather than impose available stringent 
enforcement measures. This was of particular concern as one-third of violations found by 
ministry inspectors were repeat violations and the Ministry identified significant violations in 
31% of the inspections it conducted. 


* The Ministry usually learned of contaminated sites only after serious harm to the 
environment had already occurred. 


* Over $10 million in fines had accumulated over many years. The Ministry had not 
supported the collection of fines as it could have by aggressively enforcing environmental 
legislation that allows it to suspend violators’ operations if necessary. 


We also concluded that the Ministry was not measuring and reporting on its performance in a 
comprehensive and objective manner in order to demonstrate its progress in managing the 
environment. 


Accordingly, we made a number of recommendations for improvement and received 
commitments from the Ministry that it would take corrective action. 
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3.07 Ministry of Finance 
Retail Sales Tax Program 


The Retail Sales Tax Act currently imposes a general sales tax of 8% on the retail price of 
most goods and services sold to final consumers. Various tax exemptions cover thousands of 
items and are aimed at reducing tax regression or promoting economic or social objectives. 
Examples of such exemptions include children’s clothing, equipment for use by people with 
disabilities and goods purchased by Status Indians under certain conditions. 


At December 31, 1999, approximately 380,000 vendors were registered to collect and remit 
retail sales tax (RST) to the province. RST receipts for the 1999/2000 fiscal year totalled 
approximately $12.6 billion, net of $159.5 million in refunds, which represented 21% of the 
province’s total revenue. 


Although RST revenues have increased significantly, and the Ministry has made improvements 


to its administration of this program since we last audited it in 1995, we concluded that the 
Ministry needed to further improve its procedures because it did not: 


¢ conduct research into the underground economy in order to identify sectors of the economy 


in need of more rigorous compliance and enforcement action; 


¢ have adequate procedures in place to ensure that all vendors that should have been 
registered with the Ministry to collect RST were in fact registered; 


¢ include all segments of the small vendor population in its audit coverage; 


¢ — select vendors for audit more representatively with a view to encouraging broad-based 
voluntary compliance; 


¢ follow up on all overdue vendor sales tax returns on a timely basis; and 


¢ often make adequate or timely collection efforts, particularly for the many smaller 
outstanding balances. 


We made a number of recommendations for improvement and received commitments from the 


Ministry that it would take correction action. 


3.08 Ministry of Health and Long-Term Care 

Community Health Centre Program 
Community health centres (CHCs) provide primary health care, health promotion and other 
health, educational and social services to identified priority groups within their geographical 
areas. Unlike most primary health care providers, which are funded on a fee-for-service basis, 
CHCs have fixed budgets and provide services using salaried staff. For the 1999/2000 fiscal 
year, the Ministry provided approximately $87 million to fund 56 CHCs. 


Our major concerns with the Program were: 


¢ The Ministry had not assessed the efficiency, effectiveness and ability of CHCs to provide 
quality care. 


* Funding for CHCs was not linked to the expected amount of services to be provided, the 
number of clients to be served, or the anticipated outcomes. 


3 Se eee eee 


Special Report: Accountability and Value for Money 





¢ The Ministry had no assurance that CHCs regularly review the quality of care they provide 
and the services they deliver. 


We made a number of recommendations for improvement and received commitments from the 
Ministry that it would take corrective action. 


3.09 Ministry of Health and Long-Term Care 
Emergency Health Services 


Under the Ambulance Act, the duties and powers of the Minister of Health and Long-Term 
Care include ensuring “the existence throughout Ontario of a balanced and integrated system of 
ambulance services and communication services used in dispatching ambulances.” The 
deadline for municipalities to assume responsibility for providing land ambulance services is 
January 1, 2001. During the 1999/2000 fiscal year, Emergency Health Services’ expenditures, 
prior to recoveries from municipalities for their portion of ambulance operating costs, were 
approximately $404 million. 


Our major concerns were: 


¢ Land ambulance services were being downloaded to municipalities at a time when over 
50% of land ambulance operators were not meeting response time requirements, which 
were based on 1996 actual response times. In addition, these requirements varied widely 
across the province. 


¢ The Ministry estimated that an additional $40 million annually and $11.6 million in one-time 
funding were needed to meet established response time requirements. 


* The risk of poor response times was increased because, as stated by the Emergency 
Services Working Group, 36% of the time that hospitals requested redirect consideration 
and critical care bypass, their emergency departments were not at full capacity. 


* The realigned land ambulance system may not provide a balanced and integrated system of 
services and may be more costly to Ontarians. The Ministry estimated that in the year 
2000, an additional $53 million would be needed to maintain the existing level of service, 
which was already not meeting response time requirements. 


We made a number of recommendations for improvement and received commitments from the 
Ministry that it would take corrective action. 


3.10 Ministry of Health and Long-Term Care 
Health Service Organization and Primary Care Network Programs 


The Health Service Organization (HSO) Program was established in 1973, and the Primary 
Care Network (PCN) Program was established in 1999. Each HSO and PCN comprises 
physicians who have agreed to provide a defined set of primary health care services to their 
enrolled patients. During the 1999/2000 fiscal year, the Ministry provided transfer payments 
totalling approximately $75 million to HSOs and $11 million to PCNs. 


Our major concerns with the programs were: 


* _HSO patient rosters had only been verified once, despite the fact that approximately 8,000 
of the 18,000 patients verified at that time proved to be ineligible and were removed from 
HSO rosters. 
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* The Ministry had not assessed whether it was receiving value for money for the more than 
$20 million in annual funding it provided to the Group Health Association. 


* Expansion of the PCN Program, to include 80% of eligible family doctors, was being 
planned while evaluations of the pilot PCNs were still not completed. 


* Capitation (per person) funding rates did not take into account factors that may affect the 
need for primary health care, such as patients’ medical histories. 


We made a number of recommendations for improvement and received commitments from the 
Ministry that it would take corrective action. 


3.11 Ministry of Health and Long-Term Care 
Ontario Midwifery Program 


The Ontario Midwifery Program was established in 1994 to fund professional midwifery 
services. For the 1999/2000 fiscal year, the Ministry provided approximately $17 million to fund 
midwifery services. 


Our major concerns with the Program were: 


* There was a lack of adequate information to determine whether the objectives of the 
Program were being met. 


* The Ministry had not assessed the cost-effectiveness of the current delivery and funding 
model for midwifery services. 


* The current process for referring midwifery clients to specialists may be creating additional 
costs for the health care system. 


We made a number of recommendations for improvement and received commitments from the 
Ministry that it would take corrective action. 


3.12 Management Board Secretariat 
Movable Assets 


Ministries’ movable assets consist primarily of office furniture and equipment, such as 
photocopiers and fax machines; information technology (IT) equipment, including desktop and 
notebook computers, printers and other peripherals; audio-visual equipment, such as televisions, 
VCRs and cameras; and motor vehicles. 


During the 1998/99 fiscal year—the last year for which this information was available—all 
ministries spent approximately $500 million on movable assets, much of which was spent on IT 
equipment in preparation for Y2K. However, the total value, type and quantity of movable 
assets on hand were not known because ministries did not keep adequate records in that 
regard. 


We conducted our audit work at five ministries and issued a detailed report to each deputy 
minister of the ministries included in our scope. In addition, since Management Board 
Secretariat (MBS) develops government-wide policies and standards for acquiring and 
managing movable assets and had entered into a number of government-wide standing 
agreements for IT equipment acquisition, we summarized in this Section the more significant 
issues addressed in the individual ministry reports. 
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We concluded that although ministries generally followed the process recommended by MBS 
for acquiring IT equipment and used the MBS standing agreements with various manufacturers 
for their equipment acquisitions, doing so did not ensure that they received value for money 
spent because: 


* communication between MBS and the ministries regarding the pricing provisions under the 
MBS standing agreements was insufficient to ensure that ministries were obtaining the best 
prices; 


* — there was no requirement for the ministries to demonstrate that the makes and models of 
equipment acquired were the most appropriate and cost-effective for their needs; and 


* — the cost effectiveness of leasing as the preferred option for most of the equipment acquired 
was not established, and the terms of the standing lease agreement were not competitively 
entered into nor was value for money achieved as a result of prepaying or overpaying 
lessors. 


We also concluded that movable assets were not adequately managed because: 


* accurate and up-to-date listings of all owned and leased movable assets were not 
maintained; 


* — the existence and efficient deployment of movable assets were not periodically verified; 
and 


* missing or underutilized assets were not followed up on a timely basis. 


We made a number of recommendations for improvement and received commitments from 
MBS and the ministries that they would take corrective action. 


3.13 Ministry of Natural Resources 
Forest Management Program 


Under the Crown Forest Sustainability Act, the Ministry is responsible for ensuring the long- 
term health of Ontario’s Crown forests. The Act provides for the division of the province’s 
productive Crown forests into management units and for the regulation of forestry planning, 
harvesting and renewal. The legislation also includes sanctions and penalties for non- 
compliance. The Ministry’s role in ensuring the long-term health of Crown forests is 
progressively becoming one of overseeing the activities of forest management companies. 


In the 1999/2000 fiscal year, the Ministry spent $70.8 million on forest management and 
collected stumpage charges totalling $155.7 million. In addition, two trusts set up to reimburse 
forestry management companies for renewal expenditures paid out an additional $104.8 million 
for forest renewal and related activities. 


We concluded that the Ministry did not have sufficient information to adequately meet its 
obligation to annually report on the management of Ontario’s Crown forests. In addition, the 
Ministry has not yet completed its transition from directly managing many aspects of forestry to 
implementing appropriate oversight and monitoring procedures to ensure that forestry 
companies comply with legislation and ministry policy and to ensure that the long-term health of 


Ontario’s Crown forests is managed with due regard for economy and efficiency. Specifically, 
we noted: 
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* The Ministry had not reported annually on the management of Ontario’s Crown forests as 
required by the Environmental Assessment Board. In addition, sufficient information was 
unavailable in some forest management units to properly assess the harvest area 
successfully renewed, which is a key measure of forest sustainability. 


* Over half the district offices reported that forest management companies had significantly 
over- or under-harvested. The actual harvests over the last six years for each of the 68 
management units ranged from 20% to 122% of planned levels. 


* In areas where the Ministry continued to perform compliance inspections after the 
responsibility for such inspections had been delegated to the forest management companies, 
ministry inspectors found significantly more violations than industry inspectors. 


Ministry district offices were inconsistent in imposing penalties for non-compliance and 
stricter penalties were likely warranted where warnings and less severe measures were not 
having the desired deterrent effect. 


Accordingly, we made a number of recommendations to improve the Ministry’s procedures for 
ensuring the long-term health of Ontario’s Crown forests and received commitments from the 
Ministry that it would take corrective action. 


3.14 Ontario Native Affairs Secretariat 


The Ontario Native Affairs Secretariat works with First Nations, Aboriginal organizations and 
businesses to build strong, prosperous and self-reliant Aboriginal communities. The Secretariat 
conducts land claim negotiations on behalf of the province, implements land claim settlements, 
provides core and capital funding for Aboriginal organizations and projects, and fosters 
Aboriginal economic development. The Secretariat’s expenditures were $18.7 million for the 
1999/2000 fiscal year. 


We found that although the value of land claim settlements was adequately supported, 
improvements were needed in the timeliness of reporting and accountability by First Nations for 
the use of funding provided to them for land claim negotiations. 


For the two capital programs funded by the Secretariat but delivered by two other ministries, 
we found that secretariat monitoring was inadequate and that the arrangement whereby the 
Secretariat funded but ministries delivered these programs blurred the lines of accountability. 


In its advisory role, the Secretariat helps coordinate Aboriginal-specific programs delivered by 

other ministries. Expenditures for these programs exceed $370 million annually. We concluded 

that the Secretariat needed to improve the timeliness of, accessibility to and level of detail in its 
database of information on these programs. 


Accordingly, we made a number of recommendations for improvement and received 
commitments from the Secretariat that it would take corrective action. 


3.15 Ministry of Transportation 
Monitoring School Purpose Vehicle Safety 


In conjunction with our audit of pupil transportation grants provided to school boards by the 
Ministry of Education (see Section 3.05), we determined that it was important to also consider 
the Ministry of Transportation’s role in ensuring that pupil transportation is safe. 
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We concluded that the Ministry can and should strengthen its systems and procedures for 
ensuring that operators of school purpose vehicles comply with legislative and regulatory safety 
requirements. In particular, the Ministry had not captured the information needed to ensure that: 


¢ all school buses were subject to being selected for inspection; and 


* — those operator facilities and inspection stations posing the highest risk of non-compliance 
were selected for audit. 


We also concluded that the Ministry had not sufficiently communicated the nature, extent and 
results of its enforcement activities to school boards and needed to coordinate efforts with them 
so that all safety risks are addressed and appropriate actions taken. 


We made a number of recommendations for improvement and received commitments from the 
Ministry that it would take corrective action. 
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CHAPTER TWO 


Towards Better 
Accountability 





In all previous years, I have used Chapter Two of my report to address specific issues of 
governance and accountability in government. This year there are two issues that I believe 
warrant discussion to improve accountability to the Legislature for the prudent use of public 
funds: 


* continued concerns regarding public accountability for the Ontario Innovation Trust; and 


* — legislative proposals for better public accountability. 


ONTARIO INNOVATION TRUST 


As mentioned in our 1999 Annual Report, during the 1998/99 fiscal year, the province 
established the Ontario Innovation Trust and provided it with $250 million for the purpose of 
increasing the capability of Ontario universities, colleges, hospitals and other non-profit 
organizations to carry out scientific research and technology development. 


In the 2000 Ontario Budget, the Minister of Finance announced a tripling of funding to the 
Trust with an additional endowment of $500 million for research infrastructure, including cancer 
research facilities. 


Cabinet approved the $250 million and the $500 million endowments in March 1999 and March 
2000 respectively. In both cases, the endowments were announced in the budgets pertaining to 
the subsequent fiscal periods. Because the initial $250-million payment to the Trust was in 
accordance with the accounting rules established by the Public Sector Accounting Board of the 
Canadian Institute of Chartered Accountants, we accepted it being classified as a transfer 
payment in the accounts of the province for the 1998/99 fiscal year. Specifically, the following 
circumstances were all in place before the end of that fiscal year: 


* approval by Cabinet to provide the Trust with a one-time contribution of $250 million; and 


¢ the establishment of an independent Trust (not controlled by the government) with an 
irrevocable trust agreement in place. 


In addition, as required, payment to the Trust was made shortly after it was established. 


Regarding the additional $500 million endowment to the Trust announced in the May 2000 
Budget, we concluded that the transaction had the same characteristics as the previous year’s 
endowment and accordingly, we accepted recognition of the $500 million as a fiscal 2000 
transfer payment. 


Notwithstanding the fact that the accounting treatment of the endowments to the Trust 
technically adhered to government accounting rules, I have some concerns about the 
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consequences of this treatment. For instance, it allowed the government to report that 

$500 million was spent on “innovation” in the year ended March 31, 2000, when, according to 
the 2000 Ontario Budget, only $161 million of the initial $250 million contribution had been 
approved for matching funding by the Trust, of which only $2.5 million was disbursed for 
eligible projects in the year ended March 31, 2000. In fact, the government used the Trust to 
significantly exaggerate its spending on “innovation” in that year. 


We understand that at least one other jurisdiction is currently researching if a trust, such as the 
Ontario Innovation Trust, can be regarded as operating at arm’s length from the government 
when the government, as the sole funder, effectively controls it. We will follow up on any 
developments that may result from that research. 


The Trust Agreement calls for the Ontario Innovation Trust to be administered by a seven- 
member board. The board is composed of three individuals appointed by the Lieutenant 
Governor in Council, and the remaining four appointed by: the Council of Ontario Universities 
(2), the Ontario Hospital Association (1) and the Association of Colleges of Applied Arts and 
Technology of Ontario (1). 


An independent corporation was appointed Trustee and provides services in accordance with 
the terms of the Trust Agreement. Neither the Trustee nor the employees or agents of the 
Trustee are considered to be an agent, employee or partner of the Trust’s sponsor (the Minister 
of Energy, Science and Technology) or the board. The Trust Agreement provides for an annual 
audit by an independent third party retained by the Trust’s board. 


In my 1999 Annual Report, | raised concerns about the Trust with respect to public 
accountability. In view of the fact that the government has now flowed a total of $750 million in 
multi-year program funding to an arm’s length entity, I continue to have concerns regarding 
public accountability. Specifically, I am concerned about: 


¢ — the inability of the government and the Legislature to obtain assurance that the Trust is 
spending public funds prudently and for the purposes intended and to take corrective action 
if it is not; 

* — the lack of ministerial accountability for the Trust’s activities; and 


* the fact that as the province’s Legislative Auditor, I am not permitted to conduct value for 
money audits of the Trust or inspection audits of the beneficiaries of the Trust’s grants. 


I also have concerns about the loss of interest income for the taxpayer because for the fiscal 
year ended March 31, 2000, the Trust earned $11.2 million in interest revenue. That is, the Trust 
earned interest that could have been available to the taxpayer. Moreover, $500 million was 
preflowed to the Trust well before the Trust actually required the funds. 


As a result of these deficiencies, we believe there is a significant impairment of accountability 
to the Legislature for the expenditure of public funds. 


I am of the view that any transfer payment funding should include provisions for full public 
accountability, including performance reporting and a better audit regime. This would permit the 
Legislature to evaluate what was accomplished with the funding provided and to have the 
ability to recommend what, if any, corrective actions need to be taken. 
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LEGISLATIVE PROPOSALS TO IMPROVE PUBLIC 
ACCOUNTABILITY 


STATUS OF RECOMMENDATIONS FOR AMENDMENTS TO 
THE AUDIT ACT 


On August 17, 2000, I wrote the following letter to the Honourable Ernie Eves, Minister of 
Finance: 


Dear Mr. Eves: 
Subject: Status of Recommendations for Amendments to the Audit Act 


Since 1990 the Standing Committees on Public Accounts have been voicing 
their desire to give the Provincial Auditor the legislated access to the information 
necessary to carry out full scope value for money audits of certain organizations 
receiving government grants. 


In October 1992, in the publication, A Blueprint for Learning in Ontario, Mr. Mike 
Harris then as leader of the Ontario Progressive Conservative Caucus stated: “as 
recommended by the Standing Committee on Public Accounts, the Provincial 
Auditor should be allowed to perform value-for-money audits of ALL government 
agencies and recipients of government funds.” 


As you will recall, the Standing Committee on Public Accounts considered and 
held public hearings on the subject of amendments to the Audit Act in 1996, 
which resulted in the Committee unanimously adopting the following motion: 


That the proposed amendments be provided to the Minister of 
Finance and that the Committee requests a response and action plan 
from the Minister of Finance by the Committee’s first meeting 
following the Summer recess [September 26, 1996]. 


In a letter to the then Committee Chair dated September 26, 1996, you 
responded in part as follows: 


The draft bill to amend the Audit Act as developed by the Provincial 
Auditor, in consultation with the Office of the Legislative Counsel, 
represents a significant step towards the fundamental reform of the 
public sector accountability system and | agree with the principles 
upon which it is based. 


/ concur with the proposed amendments dealing with the 
administrative changes to modernize the Act and to have the Auditor 
express an opinion as to whether the province’s financial statements 
are presented fairly in accordance with the accounting principles 
which the Canadian Institute of Chartered Accountants has 
recommended for governments. This requirement is consistent with 
the direction taken by the government in response to the 
recommendations of the Ontario Financial Review Commission. 


With respect to the amendments affecting the auditing of transfer 
payment recipients, it should be noted that a number of initiatives 
are underway, including the Who Does What discussions, which may 
result in a significant restructuring of the nature and magnitude of the 
province’s transfer payment arrangements. It may be more 
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appropriate to assess needed changes to the auditing of transfer 
payments following this restructuring. 


| then met with you on October 2, 1996 to discuss this subject and the 
timeframe for introducing a bill to amend the Audit Act, given the possible 
restructuring in the transfer payment area because of the “Who Does What” 
(WDW) discussions. In this regard, you indicated to me that it was your 
preference to await the outcome of the WDW restructuring exercise before 
considering possible amendments to the Audit Act. The WDW restructuring has 
been underway for over three years and | believe amendments to the Audit Act 
should not be postponed any longer. 


As you know, provincial monies flowing to grant recipient organizations continue 
to represent the single most significant fiscal demand on the province’s Treasury 
with about 50 percent of total government expenditures flowing to grant recipient 
organizations. 


Under the current Audit Act, the Provincial Auditor may carry out only financial 
and compliance audits of grant recipients to determine whether the recipients 
were using the grants for the intended purposes. The primary objective of the 
proposed amendments was and still is to provide the Provincial Auditor with the 
discretionary authority to perform value for money audits of organizations, such 
as community colleges, universities, hospitals, municipalities and school 
boards, which receive grants from the Province of Ontario or from an agency of 
the Crown. In this way, the other 50 percent of government expenditures will also 
be subject to value for money audits by the Legislative Assembly’s auditor, the 
Provincial Auditor, thereby strengthening public accountability. 


As you requested | am the Special Advisor to the new Ontario Financial Review 
Commission and | am aware that the mandate of the Commission includes: 
examining and reporting to you on options for improving the financial 
management, decision-making and reporting practices of the government's key 
transfer partners and | look forward to the Commission's recommendations on 
these matters. | have discussed with the Commission the existence of the 
proposed amendments of the Audit Act and the commissioners and | have 
agreed that it would be outside the mandate of the Commission to comment or 
make recommendations on these amendments. This is a matter for my Office to 
resolve with you directly. 


Because it has now been more than four years since the Standing Committee 
on Public Accounts’ motion referred to above and three years since the WDW 
restructuring exercise began, | would like to provide an update on the subject of 
this letter in my upcoming Annual Report to the Legislative Assembly. For this 
reason, | would appreciate receiving any views or comments you may have on 
the subject if at all possible by September 8. Alternatively, should you wish to 


discuss this subject in more detail, | would be pleased to meet with you at your 
convenience. 


Sincerely, 


Erik Peters, FCA 
Provincial Auditor 


When this Special Report was finalized for publication in mid-October 2000, no response to my 
letter of August 17, 2000 had been received from the Minister. However, the Deputy Minister 
of Finance advised us that the Ministry is examining a full range of accountability issues, of 
which the proposed changes to the Audit Act form a part. 
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THE PROPOSED PUBLIC SECTOR ACCOUNTABILITY ACT 
In his 1997 Ontario Budget, the Minister of Finance stated: 


To improve accountability in the public sector, the Government will introduce the 
Public Sector Accountability Act. This Act will require that public sector 
organizations: 


¢ report their financial activities in accordance with the 
recommendations of the Canadian Institute of Chartered Accountants; 


* adopt policies that ensure that the private sector has an open opportunity to 
compete to provide services to their organizations; and 


* adopt and publicly report on organizational performance using private and 
public sector benchmarks. 


On June 29, 1999, the Ministry of Finance provided the following update on the status of the 
proposed Public Sector Accountability Act: 


During the past two years the Ministry of Finance has conducted a number of 
consultations with several key stakeholders on initiatives the government could 
take to improve public sector accountability including the implementation of a 
legislated accountability framework. Several important issues and questions 
arose out of these consultations. These issues included the best means of 
supporting ongoing improvements in accountability and governance practices, 
incentives to encourage increased accountability and the logistics of 
implementing enhanced accountability. To pursue these questions, the Ministry 
of Finance co-sponsored a symposium with the Canadian Comprehensive 
Auditing Foundation that brought together a small group of leading thinkers in 
the area of public sector accountability including the Provincial Auditor. An 
initial review of the thoughtful discussion that took place at this symposium has 
provided the Ministry of Finance with a number of options which could support 
ongoing improvement in public sector accountability. 


Further research into these and other options will be completed over the next 
year. The results of this review will help the government determine how best to 
proceed in supporting ongoing improvements in public sector accountability. 


In this latter regard, as Special Advisor to the Ontario Financial Review Commission, which 
was re-established by Order in Council 2430/99 dated December 23, 1999, I am aware that its 
mandate includes examining and reporting to the Minister of Finance on options for improving 
the financial management, decision-making and reporting practices of the government’s key 
transfer partners. 


I look forward to the Commission’s report on the results of its work, which is expected in the 
fall of 2000. 


CONCLUSION 


I continue to believe that the proposed Audit Act amendments are directly related to enhancing 
public sector accountability, particularly of recipients of provincial grants. In his September 26, 
1996 letter to the Standing Committee on Public Accounts, the Minister of Finance indicated 
that the draft bill to amend the Audit Act represents a significant step towards the fundamental 
reform of the public sector accountability system and that he agrees with the principles upon 
which it is based. 
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It has now been more than four years since the Public Accounts Committee unanimously 
endorsed our proposed amendments to the Audit Act without action on the part of the 
government. I believe that the proposed revisions to the Audit Act deserve more serious and 
timely consideration. Accordingly, I recommend that the Committee revisit this subject again in 
its upcoming sitting. The sole reason for proposing the amendments to the Audit Act is to 
enable my Office to better and more comprehensively serve the Legislature and thereby the 
taxpayers of Ontario. 


With regard to the proposed Public Sector Accountability Act, as previously stated, I continue 
to be a strong advocate of any legislation that enhances accountability in the public sector. With 
this in mind, I look forward to the recommendations of the Ontario Financial Review 
Commission. 
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MINISTRY OF AGRICULTURE, FOOD AND 
RURAL AFFAIRS 


AgriCorp 





BACKGROUND 


AgriCorp, a corporation without share capital, is an agency of the Ontario Ministry of 
Agriculture, Food and Rural Affairs. Established by the AgriCorp Act, 1996, its objectives are 
to administer plans of crop insurance as well as other programs related to the agriculture and 
food industries. The most significant of these activities is the administration of the Ontario Crop 
Insurance Fund and the Market Revenue Program. 


The Ontario Crop Insurance Fund, established in 1966 and formerly administered by the 
Ontario Crop Insurance Commission, operates under the Crop Insurance Act (Ontario). Crop 
insurance provides farmers with financial protection against yield reductions due to natural 
events such as drought, flooding and plant disease. Farmers pay half the cost of crop insurance 
premiums, and the provincial and federal governments pay the other half. 


The Market Revenue Program was established by an agreement between the governments of 
Canada and the provinces and has been in operation since April 1, 1991. The Program protects 
farmers against reduced income caused by low commodity prices for certain designated crops. 
The provincial and federal governments share premium funding for this program equally. 


The administrative costs of operating the Ontario Crop Insurance Fund and the Market 
Revenue Program are funded equally by the province and the federal government. The 
Corporation maintains a General Fund through which all administrative expenses and associated 
government revenue contributions flow. On a cost recovery basis, a number of other activities 
are also funded through the General Fund, such as the Ontario Whole Farm Relief Program. 
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Summary of Fund Balances and Activities for the Year Ended March 31, 2000 
($ millions) 
























Ontario 
Crop 
Insurance 















Market 
Revenue 
Fund Program 


164 | 862 1295 
(ies) | 83) | 142.0 | (1968) 
Fund Income (Loss) (0.1) (115.1) (67.3) 


Assets 33.9 332.7 237.4 | = 604.0 | 0 
Liabilities (32.5) (10.1) (64.4) -- Se 0) 


AgriCorp 
Total 


General 
Fund 


















Fund Balance 1.4 322.6 173.0 497.0 





Source: Audited Financial Statements 


The Corporation is governed by a volunteer board of directors appointed by the Lieutenant 
Governor in Council. The board is made up of 16 representatives of the agriculture community 
and a ministry representative. Board members also make up the crop insurance sub-committee 
and the audit and finance sub-committee, as well as an executive board that is responsible for 
AgriCorp’s overall operation. 


AUDIT OBJECTIVES AND SCOPE 


The objectives of our audit of AgriCorp’s operations were to assess whether adequate 
procedures were in place: 


* to ensure compliance with legislation and corporate procedures; 
* to ensure that resources were managed with due regard for economy and efficiency; and 


* to measure and report on the Corporation’s effectiveness in fulfilling its legislated 
responsibilities. 


The criteria used to conclude on our audit objectives were discussed with and agreed to by the 
Corporation’s management. 


Our audit was performed in accordance with the standards for assurance engagements, 
encompassing value for money and compliance, established by the Canadian Institute of 


Chartered Accountants, and accordingly included such tests and other procedures as we 
considered necessary in the circumstances. 


The scope of our audit, which was substantially completed by January 2000, included 
discussions with staff and a review and analysis of documentation at the Corporation’s head 
office. We surveyed AgriCorp’s board of directors by means of a questionnaire. We also 
reviewed governance structures and crop insurance programs in other Jurisdictions. 


eee 


24 Office of the Provincial Auditor 





3.01 





Internal audit services are to be provided to AgriCorp by the Ministry of Agriculture, Food and 
Rural Affairs. However, because no internal audit reports had been issued on AgriCorp since 
its inception on January 1, 1997, we were unable to reduce the extent of our audit work. 


OVERALL AUDIT CONCLUSIONS 


AgriCorp did not have adequate procedures in place to ensure that its activities complied with 
legislation and corporate procedures. In addition, AgriCorp failed to manage certain of its 
resources with due regard for economy and efficiency and, on a number of occasions, failed to 
safeguard funds entrusted to it. Also, AgriCorp did not have the necessary governance and 
accountability procedures in place to ensure that the Corporation was well managed or to 
provide the information required to measure and report on its effectiveness in fulfilling its 
legislated responsibilities. Some of our major findings were as follows: 


¢ AgriCorp undertook a speculative investment initiative to buy and sell bonds on a daily basis 
in the hope of earning a quick return on the fluctuation of interest rates. This two-week 
initiative resulted in a loss totalling $325,000 to AgriCorp’s General Fund. 


¢ AgriCorp has sought to utilize money from the Ontario Crop Insurance Fund to pay for its 
administrative expenses which corporate management contended was acceptable practice 
in the insurance industry. However, legislation prescribes, and a legal opinion confirmed, the 
allowable uses of the Fund, which do not include administrative expenses. Such expenses 
are to be paid from AgriCorp’s General Fund. 


¢ AgriCorp needed to improve procedures to ensure that the amounts of crop losses claimed 
were accurate and in compliance with corporate procedures. For example, individuals were 
allowed to pay insurance premiums after their crops had already been harvested. 


¢ Information technology development was poorly planned, controlled and managed. Since 
January 1997, AgriCorp has initiated various information technology projects without clear 
direction, resulting in expenditures of at least $3 million for which it received little or no 
value. 


¢ AgriCorp engaged several information technology consultants for a number of years 
without competition. In addition, at the taxpayers’ expense, AgriCorp sent consultants on 
training courses, one of which cost more than $10,000. Consultants should only be engaged 
if they can bring the necessary skills to the job. 


¢ AgriCorp engaged an investment advisor, without competition, for a minimum annual fee of 
$400,000. The advice received was of little value as the advisor repeatedly recommended a 
diversified portfolio that included investments the Corporation was legislatively prohibited 
from purchasing. 


* Through a private insurance intermediary, AgriCorp paid $14.1 million for reinsurance 
coverage over a two-year period. The intermediary was engaged without competition. 
Also, AgriCorp had not prepared a thorough cost/benefit analysis of the potential benefits 
of reinsurance. 
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¢ Both employees and managers often disregarded travel expense procedures as travel 
claims were approved and paid without proper receipts, which, in some cases, resulted in 
duplicate payments. Additionally, the travel expense claim process was inappropriately used 
to purchase major items, thus by-passing purchasing department controls. 


* AgriCorp has adopted six considerably different vision statements in the past three years, 
an indication that it has no clear understanding of its role or objectives. Without such an 
understanding, it cannot develop measures to assess the achievement of its legislated 
responsibilities. 


* AgriCorp had established a governance structure designed to help direct and manage its 
affairs. However, members of its board of directors often lacked the necessary information 
to make informed decisions, and, as noted by a consultant engaged by the board, 
management provided information to the board in a fashion that appeared to be “forcing or 
manipulating a decision.” 


DETAILED AUDIT OBSERVATIONS 


COMPLIANCE WITH LEGISLATION AND 
CORPORATE PROCEDURES 


INVESTMENTS 


Legislation restricts all of AgriCorp’s investments to highly liquid, high-grade money-market 
instruments, such as federal and provincial bonds, deposit notes issued by domestic financial 
institutions and other securities approved by the Minister of Finance. As of March 31. 2000. 
AgriCorp’s investments totalled $355 million. AgriCorp does not directly manage the money in 
the Market Revenue Program fund because, as required by an order in council, this fund is to 
remain on deposit with the Ministry of Finance. 


In September 1999, AgriCorp used money from the General Fund to embark on a pilot project 
to buy and sell bonds on a daily basis in the hope of earning a quick return on fluctuating 
interest rates. However, the project was terminated after two weeks because of losses totalling 
$325,000. 


At the time AgriCorp stopped its daily trading initiative, it had a Government of Canada bond 
that it had purchased for $6.5 million. The fair market value of the bond subsequently declined 
to $6.2 million, and the bond and its associated losses were inappropniately transferred from the 
General Fund to the Ontario Crop Insurance Fund. After we brought this matter to the attention 
of management in January 2000, the Ontario Crop Insurance Fund was reimbursed by the 
General Fund and all losses were assumed by the General Fund. 


AgriCorp also inappropriately used funds held for the Ministry of Agriculture, Food and Rural 
Affairs in its daily trading strategy. At the time the $6.5 million bond was purchased, AgriCorp 
only had $3.6 million of its own money in the General Fund. The remaining $2.9 million came 
from funds held for the Ministry to make payments under the Ontario Whole Farm Relief 
Program. The agreement between AgniCorp and the Ministry stated that the investment policy 
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applicable to these funds would be one that was risk-averse and limited to investments in 
Canadian money-market and Canadian fixed-income instruments. On another occasion, in May 
1999. AgriCorp invested Ontario Whole Farm Relief Program Funds in a long-term bond that 
was sold in September 1999 for a loss of $61,000. By using these funds inappropriately, 
AgriCorp violated its fiduciary responsibility. 


AgriCorp has a buy and hold to maturity strategy for long-term investments in the Ontario Crop 
Insurance Fund. These investments are to be sold if there is a need to meet indemnity 
obligations. However, in June and October 1999, AsniCorp sold a total of three long-term bonds 
for $19.5 million. AgriCorp had purchased these bonds for $20.7 million for a total capital loss 
of approximately $1.2 million. The proceeds of the sale were reinvested in similar securities, so 
over the life of the new securities, up to 27 years, the Fund will eventually recoup these losses. 
However, at the time of the sale, there was no need to sell these investments to meet indemnity 
obligations. 

In the use of the resources entrusted to it, cited above, AgriCorp acted without proper 
authorization or adequate monitoring and reporting. In this connection, we noted the following: 


¢ The Corporation’s board had not approved the speculative trading strategy. We were 
informed that management had obtained verbal approval to proceed from two of the three 
members of the board’s audit and finance committee. However, this committee functions in 
an advisory capacity only and did not have the authority to authorize the strategy. 

« AgriCorp had both a strategic and a tactical committee, comprising board members and 
management representatives respectively, to monitor investment policy and trading activity. 
However, monitoring was ineffective since neither of these committees ever met regarding 
the speculative daily trading activity. 


Recommendation 


To ensure that proper controls are in place to safeguard investments, 

AgriCorp should: 

* engage only in investment strategies that comply with corporate 
procedures and the requirements of formal agreements; and 


e establish and ensure compliance with appropriate procedures to monitor 
investment activities. 





Corporation Response 


AgriCorp acknowledges that poor judgment was exercised in connection with 
the daily trading pilot project. 


_ Over the two-ana-one-half year period since AgriCorp put in place an 
investment strategy for the Ontario Crop Insurance Fund, investment results 

_ have modestly exceeded the external benchmark for all similar Canadian 

_ bond portfolios. AgriCorp has managed the Fund to earn a competitive rate 

of return for its stakeholders. 
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The AgriCorp board of directors approved a general fund investment strategy 
at its meeting of March 27, 2000 that complies with all the requirements of the 
AgriCorp Act, 1996. Investment duties have been shifted to the AgriCorp 
Finance Department. The AgriCorp board has assumed the investment 
monitoring duties of the tactical and strategic investment committees. The 
board has established new procedures and controls and tightened existing 
ones. 


FUND ADMINISTRATION 


AgriCorp administers the Ontario Crop Insurance Fund’s $320 million, which is invested in 
income earning securities. The Fund is made up of crop insurance premium receipts from 
producers, the province and the federal government as well as accumulated interest on its 
investments. Since its inception, the Fund has generated interest income of more than 

$125 million. The only expenditures that can be paid out of the Fund are insurance contract 
indemnity payments, the repayment of any loans and reinsurance expenses. 


The federal and provincial governments share equally the costs of administering the crop 
insurance program. Their contributions are deposited in AgriCorp’s General Fund, from which 
administrative expenses are to be paid. AgriCorp and the Ministry annually negotiate a budget 
for administrative expenses, and the Ministry has instructed the Corporation to operate within 
that budget. 


According to a Memorandum of Understanding with the Ministry, AgriCorp is responsible for 
maintaining the integrity of each fund that it administers. However, primarily due to budget 
pressures, AgriCorp has sought to utilize interest earnings from the Ontario Crop Insurance 
Fund to pay for its administration expenses. For example: 


* Inthe 1997/98 fiscal year, AgriCorp proposed to remove the $10.1 million of interest and 
investment income earned during the year by the Ontario Crop Insurance Fund and transfer 
this money to the General Fund. AgriCorp argued that the Ontario Crop Insurance Fund 
was not entitled to the interest earned from its investments. 


¢ AgriCorp charged investment administration costs to the Ontario Crop Insurance Fund. 
Administration costs are a federal-provincial government responsibility and are not to be 


paid from funds that have accumulated to indemnify Ontario farmers for their future crop 
losses. 


* Pursuant to an agreement between AgriCorp and the federal and provincial governments, 
the Corporation also charged information technology expenses to the Ontario Crop 
Insurance Fund. By charging administrative costs to the Ontario Crop Insurance Fund, 
AgriCorp would forgo millions of dollars worth of federal-provincial funding that Ontario 
farmers would eventually have to pay. This is because administration costs are funded 
50-50 by the two levels of government while funding for crop insurance is shared 25-25-50 
among the federal and provincial governments and Ontario farmers respectively. 


AgriCorp presented legal justification for the removal of the interest earned by the Ontario 
Crop Insurance Fund and for charging administration costs to the Fund. Consequently, we 
requested that outside counsel be engaged to review AgriCorp’s interpretation of the legislation. 
The resulting legal opinion clarified the position that Ontario Crop Insurance Fund interest 
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cannot be used for administrative purposes. Consequently, due to the transactions previously 
noted, as of March 31, 1999, the General Fund owed the Ontario Crop Insurance Fund a total 
of $2.1 million. Since the legislation allows for interfund loans, interest should be paid on these 
interfund transactions. Monies intended for crop insurance should not be used to finance the 
operations of AgriCorp. 





Recommendation 





To ensure the integrity of all funds under its stewardship and to ensure that 
no fund benefits at the expense of another, AgriCorp should: 
¢ ensure that the Corporation’s activities are legislatively sound; 


¢ implement policies for interfund transactions, including the payment of 
fair rates of interest on any interfund loans; and 


e review administrative funding arrangements to properly deal with funding 
pressures. 
Corporation Response 


Since AgriCorp was established, management has sought clarification of the 
legislation concerning the use of Crop Insurance Fund interest earnings. 





In 1998/99, in good faith, AgriCorp obtained legal advice and entered a three- 
way agreement between itself, the Ontario Ministry of Agriculture, Food and 
Rural Affairs, and the Department of Agriculture and Agri-Food Canada, 
allowing Crop Insurance Fund access for one specific purpose for a defined 
period of time. AgriCorp accessed the Fund under the provisions of the three- 
way agreement for $1.4 million. An additional $0.7 million was deducted for 
external investment management. AgriCorp subsequently accepted outside 
counsel's position and has returned a portion of the funds and made 
provision to return the balance with interest. AgriCorp has made provision 
for the payment of fair rates of interest on interfund transactions and will 
formally document the policy. 





In future, AgriCorp’s recourse will be to the Ministry and Agriculture Canada 
for administrative funding. User fees might augment the two primary 
sources. 








INSURANCE CLAIMS 


Crop insurance protects producers from reduced crop yields and losses caused by natural 
events such as drought, excessive rainfall, flood, frost, hail, insects, plant disease and wind. For 
the 1999/2000 fiscal year, over 39,000 contracts for 53 commercially grown crops provided 
$1.2 billion worth of insurance to producers. Crop insurance is generally based on the average 
yield for each crop grown by individual producers. When the harvested yield is less than the 
guaranteed yield due to an insured cause, the producer is entitled to make a claim to AgriCorp 
and be reimbursed for the difference between the two yields. 
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We found that procedures were inadequate to provide the information necessary to determine 
whether the amounts of crop losses claimed were accurate and allowed by legislation. 
Specifically, we noted: 


The Crop Insurance Act (Ontario) states that “AgriCorp shall not enter into a contract of 
insurance with a person to insure an agricultural crop or a type of perennial plant if the 
contract insures less than the entire crop...” The legislation is intended to ensure that 
producers do not divide their land and separately insure smaller units. Multiple crop 
insurance contracts would increase the probability of receiving an indemnity payment as 
high yielding areas would not be offset against low yielding areas when all areas are 
totalled to determine if an indemnity should be paid. However, for the 1999/2000 fiscal year, 
AgriCorp implemented a program termed Optional Unit Coverage, which allowed 
producers to purchase separate insurance contracts for the same crop planted in different 
locations. For the 64 producers that participated in this program, AgriCorp made claim 
payments of $439,000, which was $197,000 more than if each of these producers had 
insured crops with one contract. 


AgriCorp received a legal justification for this program stating that all crops must be 
insured, however, different factors may be applied to different units within the umbrella 
policy. Also, AgriCorp has stated that the program will not cost the Ontario Crop Insurance 
Fund more because additional deductibles are to be applied. However, in the first year, this 
program cost the Fund an additional $197,000 in indemnities for only 64 producers. Also, if 
the program is actuarially sound, it would not monetarily benefit producers and, if it is not 
actuarially sound, there is increased risk exposure to the Ontario Crop Insurance Fund. 
AgriCorp intends to offer this program again in the 2000/01 fiscal year. 


AgriCorp sets application, acreage reporting and premium payment deadlines for the 
various crops insured. In the majority of cases, the premium payment deadline is set 10 
days after planting. However, for some crops such as green beans, green peas, processing 
tomatoes, sweet corn, seed corn and flue-cured tobacco, AgriCorp allowed premium 
payments to be remitted by producers after the harvest. 


For example, a premium payment totalling $210,000 on behalf of 163 seed corn producers 
was submitted on November 25, 1999, well after the harvest. Normally, for insurance 
coverage to be in effect, premiums must be submitted before the harvest. We question the 
prudence of allowing some producers to pay their premiums after the harvest, especially in 


light of the fact that AgriCorp was enforcing the premium deadlines for producers of other 
crops. 


In June 1999, the board’s crop insurance committee passed a motion to cancel the 
insurance contracts of forage producers who did not report acreage and pay their 
premiums by the deadline date. Staff attempted to enforce the premium deadline of May 1, 
1999. On July 12, 1999, AgriCorp informed the delinquent producers that their insurance 
coverage for 1999 was cancelled for not remitting the premium before the deadline date. 


Because it received complaints about the deadline enforcement, the crop insurance 
committee instructed staff to personally visit the producers whose insurance had been 
cancelled and offer them another opportunity to insure their 1999 crops. Of the producers 
contacted, 28 took insurance, which eventually resulted in AgriCorp paying 17 producers’ 
claims totalling $40,000. This undermines the enforcement efforts of staff and is contrary to 
the committee’s own approved policy for insurance cancellation. 
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¢ The number of insured acres is one factor used to calculate the amount of loss and the 
corresponding indemnity payment. We noted that AgriCorp did not require claims adjusters 
to actually measure the acreage of a damaged crop prior to settling a claim. Adjusters 
visually inspected damaged crops and determined on that basis whether a producer’s 
reported damage was reasonable. In 1999, AgriCorp kept track of the reasons for any 
indemnity claim adjustments. In the 38 cases where adjustments were made due to acreage 
being measured, the claims amount paid to producers was reduced by $250,000. This 
illustrates the importance of actually measuring the acreage for damaged crops. 


¢ Ifa producer does not achieve a guaranteed yield, an indemnity is normally paid for the 
difference between the actual yield and the guaranteed amount. However, we noted 
situations where producers could receive indemnities even when they had no losses. For 
example, claims for forage crops were based on a computer model that considered soil 
mixture, sunshine, temperature and rainfall information to simulate forage growth and arrive 
at an expected yield. Rainfall measurements were provided to AgriCorp by the individual 
producers. Indemnities were determined by comparing guaranteed yields to model- 
generated yields rather than actual ones. Consequently, there is a risk that producers could 
receive indemnity payments even if they had no actual loss, and actual losses could go 
uncompensated if the model-generated yields showed no losses. 





Recommendation 


To ensure that the proper indemnity payments are made in compliance with 
legislation and corporate policy, AgriCorp should: 


* manage the Optional Unit Coverage Program to ensure that there is no 
additional risk exposure to the Ontario Crop Insurance Fund; 


¢ equitably enforce the deadline dates for insurance applications and 
premium payments on all crops; 

e direct insurance adjusters to measure the acreage related to any claim for 
crop loss or damage; and 

* reassess procedures as necessary so that indemnities are paid only when 
actual yields are less than guaranteed yields. 


Corporation Response 


Optional Unit Coverage was offered on a limited pilot project basis. Premium 
rates were established with higher deductibles, so that, over many years, the 

total claim payout will not exceed what it would have been had each producer 
insured with one contract. If the project advances beyond the pilot stage, it is 

AgriCorp’s intention to manage it on an actuarially sound basis. 


The Crop Insurance Act (Ontario) states that “AgriCorp shall not enter into a 
contract of insurance with a person to insure an agricultural crop or a type of 
perennial plant if the contract insured less than the entire crop...” This is 
interpreted by AgriCorp and corroborated by ministry counsel as intending to 
prevent producers from only insuring part of their acreage, that is, high-risk 
acres. AgriCorp complied with the Act for the Optional Unit Coverage Pilot 
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| Project because customers must offer all acres of the crop for insurance 


coverage. The pilot project was undertaken with the approval of Agriculture 
Canada which monitors crop insurance plan design. 


Certain processors/commodity boards collect premium, acreage and yield 
information on AgriCorp’s behalf, thereby saving the Corporation 
administrative expense. Deadline dates are deferred for these customers; 
however, there is no opportunity for producers to opt out of paying premiums 
once enrolled. AgriCorp will review the equity of these arrangements. 


Although AgriCorp does not measure the actual acreage of every claim, it 
recognizes the importance of accurate acreage measurement. Acreage is 
reported to AgriCorp early in the growing season, long before a claim can be 
predicted. There is no incentive for producers to declare inaccurate acreage. 
Random audits on a sample basis are conducted each year to protect 
program integrity. The results of these audits will be reported to the AgriCorp 
board, which will judge the effect on integrity and may recommend increased 
measurement. 





The forage commodity is administratively difficult, and it is cost-prohibitive to 
measure an actual yield for all insured growers. Therefore, the current plan 
uses a yield simulation that provides producers coverage at a reasonable cost 
to customers and government. No crop insurance jurisdiction in Canada 
offers an intensively managed forage plan based on individual farm yields. 
AgriCorp is testing a revised forage plan that features independent rainfall 


measurements and is much simpler than the simulation. fal 





DUE REGARD FOR ECONOMY AND EFFICIENCY 


NFORMATION TECHNOLOGY MANAGEMENT 


AgriCorp’s Information Technology Department is responsible for providing reliable and secure 


C 


omputer systems and services. The Department’s functions include providing support and 


maintenance for existing business applications, development and testing of new software 
applications, and management of the information technology infrastructure. The Department’s 
expenditures for the 1998/99 and 1999/2000 fiscal years totalled $12 million. 


When AgriCorp was created in January 1997, the Information Technology Department planned 
to move the Corporation’s information technology infrastructure from the government’s 
mainframe computer to an in-house client-server technology. The development of this new 
client-server technology required considerable expertise. AgriCorp had concerns as to whether 
the project would meet its needs and, accordingly, engaged a consultant to carry out a strategic 
information technology review. The consultant found that: 


the staff working on the project lacked the in-depth technical experience to develop the 
new technology; 


management and the project team had no clear agreement concerning what the business 
requirements of this project were or how they could be met by the proposed project; and 
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¢ the project team had not developed an overall architecture for the system that identified the 
standards to be followed during construction, the components to be built or how those 
components would relate to each other. 


Despite these concerns, management decided to continue with the project and, in October 1997, 
developed a two-year strategic plan for moving its information technology infrastructure from 
the mainframe computer to an in-house arrangement. 


We reviewed this project and the other information technology development projects AgriCorp 
had initiated since January 1997. We concluded that AgriCorp had poorly managed these 
projects. We had several concerns in this regard: 


¢ We noted that, although AgriCorp prepared a two-year strategic plan for the development 
of its in-house client-server technology, it had not prepared a detailed business case to 
evaluate alternatives, assess their benefits and risks, or compare costs to ensure the 
greatest net benefits. Without a proper business case, the Corporation was not able to 
demonstrate that taxpayers would receive value for funds expended. 


¢ Instead of a structured approach to achieving predetermined milestones, management 
consistently searched for and evaluated other application systems while development work 
continued. As a result, a number of changes in direction were made which jeopardized the 
success of the project to move AgriCorp’s information technology infrastructure in-house. 


* In April 1999, AgriCorp decided to shift information technology development back to the 
government’s mainframe environment and redevelop the original software to address user 
needs. Changing direction to the mainframe environment meant that the hardware 
purchased and software developed for the in-house technology were no longer useful. As a 
result, AgriCorp spent at least $3 million for which it received little or no value. 


¢ In November 1999, AgriCorp started to develop a new information technology 
infrastructure that would allow customers to conduct their crop insurance business through 
the Internet. AgriCorp undertook this development without the benefit of a detailed 
business case. The infrastructure for this development may not be compatible with 
AgriCorp’s current environment, highlighting the need for a long-term strategic plan for 
information technology. 


¢ In addition to the projects described above, AgriCorp purchased two major software 
applications. These purchases included $60,000 for a number of financial management 
applications and a $129,000 application to connect field staff to head office. These 
applications did not provide the required functionality when installed, and consequently, the 
corporation received no value for the money spent. 





Recommendation 







To ensure that information technology project management results in the 
effective execution of plans and achieves results economically, AgriCorp 
should: 


¢ develop a long-term strategic plan for information technology that 
reflects the operational needs of the Corporation; 
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E prepare a proper business case for all potential systems development 
projects, including an assessment of corporate needs, options available 
and a detailed cost/benefit analysis of each option; 

¢ before proceeding with any system development project, develop and 
receive board approval for a project plan outlining project deliverables 
and the nature and timing of milestones to be achieved; 

¢ establish a rigorous monitoring program to ensure the achievement of 
milestones and the satisfactory completion of all system development 
projects; and 

¢ thoroughly evaluate any new software applications before purchasing 
them to ensure that they meet corporate needs and can be implemented. 


Corporation Response 


A three-year strategic plan is currently being prepared. The first draft will be 
completed by the end of September 2000. In the absence of a board-approved 
strategic information technology (IT) direction, the project to allow customers 
to conduct their crop insurance business through the Internet was cancelled. 


All IT projects are now required to have a proper business case before they 
are initiated. 


A requirement for board approval of business cases and project plans for 
projects over $100,000 has been proposed. 





A project methodology following the Project Management Institute’s 
guidelines will be followed for systems development projects. | 
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CONSULTING SERVICES 


During the 1998/99 and 1999/2000 fiscal years, AgriCorp paid over $1.8 million for consulting 
services, primarily for information technology projects. We examined a sample of consulting 
assignments and found that AgriCorp did not have adequate procedures in place to ensure the 
economic acquisition and proper management of consulting services. For example: 


* AgriCorp did not maintain an open and fair process for acquiring consulting services. None 
of the consulting contracts we reviewed used a competitive process, and AgriCorp was 
unable to provide documentation justifying or approving any exemptions from a competitive 
process. Without a competitive process, the Corporation cannot demonstrate that it obtained 
value for the funds expended. 


* We noted that numerous information technology consultants had been retained by AgriCorp 
or its predecessor organization on year-to-year contracts lasting more than five years, and 
in two cases for 13 years. In these instances, consultants worked regular hours and were 
even listed as AgriCorp employees in the government phone book. We noted that these 
consultants were often assigned on an as-needed basis to work on projects not outlined in 
their contracts. In addition, these long-term consultants were paid from $325 to $640 per 
day, which was substantially more than AgriCorp employees earned. 
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¢ We noted examples where consultants were engaged by AgriCorp and subsequently sent 
on training courses paid for by the Corporation. In one case, a consultant who had been 
retained for nine years was sent on an advanced programming practices course at a cost to 
taxpayers of $10,345. We question this practice since it is the responsibility of consultants to 
keep up-to-date with the skills necessary to complete assignments. 


¢ Upon the completion of each consulting assignment, a formal written evaluation should be 
prepared to assess the quality of the work, whether value for money was obtained and the 
suitability of the consultant for future work. AgriCorp had not prepared written evaluations 
for any of the consulting assignments we reviewed. 


Recommendation 


To obtain value for funds expended on consultants, AgriCorp should ensure 
that: 


* consultants are engaged and contracts are renewed through a 
competitive process and any exceptions are adequately justified, 
documented and approved; 

¢ consultants are not used to perform work other than that specified in 
their contracts; 

* consultants have the necessary skills to carry out their work assignments 
and not be trained at taxpayers’ expense; and 


¢ assignments are formally evaluated upon completion. 





Corporation Response 


The Corporation is well into the process of ensuring that all information 
technology staff have employee status, eliminating the need for the regular 
use of consultants. 


If consultants are hired in the future, it will be for specific purposes, and they 
will be monitored for adherence to the specified work tasks defined in the 
contract. 








INVESTMENT SERVICES 


In November 1997, AgriCorp hired an external investment advisor without using a competitive 
process. The advisor was to provide the Corporation with advice on its investments and help 
write an investment policy statement. The contract stipulated that AgriCorp was to pay a fee 
based on a rate of 0.3% of the aggregate investment value. In addition, the term of the contract 
was indefinite as an expiry date was not specified. 


During 1998, AgriCorp began to carry out some investment management services in-house and 
renegotiated the contract with the external investment advisor, again without a competitive 
process. The renegotiated contract began on December 1, 1998, and expires on December 31, 
2000. The renegotiated fees were based on a rate of 0.2% of the investment asset value, with a 
minimum annual fee of $400,000. The contract could not be terminated without paying the 
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minimum annual fee for the full term of the contract. In addition, the contract provided no 
remedies for AgriCorp in the event of poor performance by the advisor. 


We question paying such a high price for advice that is of little value. The advisor repeatedly 
recommended a diversified portfolio that would include stocks and other related instruments 
AgriCorp is not allowed to purchase under the AgriCorp Act, 1996. In February 2000, a 
consultant’s report entitled Crop Insurance Delivery Review also indicated that “AgriCorp 
could have acquired competitive investment services at substantially less [than the current 
arrangements] by re-tendering the contract.” 


The Corporation’s Memorandum of Understanding with the Ministry states that AgriCorp “shall 
manage its investments and borrowing activities under the direction of the Ontario Financing 
Authority.” The primary objective of the Authority is to assist public organizations to borrow 
and invest money. However, AgriCorp had not contacted the Authority to obtain its advice. 








Recommendation 


To ensure that necessary investment services are acquired economically, 
AgriCorp should: 

e assess the need for an investment advisor; 

e acquire investment services through an open and competitive process; 


e obtain legal advice regarding the terms and conditions of all major 
contracts; and 


¢ consider obtaining investment advice from within the government. 








Corporation Response 


On April 25, 2000, the AgriCorp board of directors approved a 
recommendation from management to let the investment counselling 
arrangement lapse. The board, recognizing funding constraints, approved in- 
house management of investments by staff experienced in insurance 
company investment procedures. AgriCorp will assess the suitability of 
investment advice from within government. 











REINSURANCE COVERAGE 


The Ontario Crop Insurance Fund has paid indemnities to producers cumulatively totalling 
$930 million over the 31 years since the Fund’s inception. Historically, AgriCorp and its 
predecessor organization, the Crop Insurance Commission of Ontario, paid indemnities due to 
crop losses from money accumulated in the Fund. In 1998 AgriCorp started a program 


designed to reduce its risk of unusually high indemnity claims by purchasing coverage from 
reinsurance companies. 


AgniCorp has a potential annual crop insurance liability of $1.2 billion, which would have to be 
paid if all insured crops completely failed in the same year. AgriCorp reinsured with private 
insurance Carriers to assume crop insurance losses that ranged between $121 million and 


$242 million, with the Fund paying indemnities above or below these amounts. With respect 
to reinsurance, we had the following concerns: 
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¢ After evaluating two reinsurance intermediaries, AgriCorp engaged one of them without 
tender. The intermediary assessed AgriCorp’s crop insurance risk, devised a reinsurance 
plan and placed various portions of the reinsurance with a number of private carriers. 
AgriCorp paid the intermediary $8 million and $6.1 million for reinsurance in the 1998/99 
and 1999/2000 fiscal years respectively. 


¢ AgriCorp prepared a proposal to analyze alternatives and outline its justification for 
acquiring reinsurance. The proposal indicated that reinsurance would provide long-term 
viability for the Fund, allow for reduced premium costs to producers and increase 
investment income by moving into longer term investments. We noted that insurance 
premium rates for producers had declined in the past two years mainly due to a reduction in 
commodity prices and indemnity claims. However, AgriCorp could not provide us with 
evidence that any portion of the rate reduction was the result of the reinsurance program. 
In addition, the Corporation had not performed an analysis to demonstrate that investment 
yields had increased because of the reinsurance. 


¢ Using the current reinsurance arrangements, we reviewed crop insurance claims for the 31 
years the program was self-insured. Reinsurance is purchased for three levels of claims 
and a level three (the highest) payout would have occurred only once, in 1979, for 
$8 million. The cost of level-three coverage was $1.2 million in the 1998/99 fiscal year. 
Although it is not possible to predict future indemnities, the complexity of reinsurance 
highlights the need for AgriCorp to obtain the advice of an objective reinsurance expert. 


¢ AgriCorp’s board was requested by management to approve the reinsurance arrangements 
after these arrangements had already been finalized with the intermediary. 


Recommendation 


To ensure that both its current reinsurance program and any future 
reinsurance arrangements are economical and appropriate, AgriCorp should: 


¢ determine whether the reinsurance program has resulted in any direct 
rate reduction to premiums or increase to investment yields; 

* consider obtaining expert advice to review the current arrangements and 
potential options; 

* prepare a business case for reinsurance that quantifies expected costs 
and benefits; 

¢ acquire any future reinsurance through a fair, transparent, competitive 
process; and 

¢ obtain board approval prior to entering into any future reinsurance 
arrangements. 








Corporation Response 


AgriCorp is committed to ensuring that all decisions regarding the 

management of the fund, including the purchase of reinsurance, are 

appropriate for the risk profile of the Ontario Crop Insurance Fund. AgriCorp 
ill prepare a business case that will thoroughly assess all viable options 


w 
before the next crop year. 
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Direct premium rate reductions did occur as the result of the purchase of 
reinsurance. Our actuary, by virtue of reinsurance, reduced the premium load 
for fund self-sustainability. As a result of purchasing reinsurance, the 
Corporation changed the investment strategy for the Ontario Crop Insurance 
Fund to lengthen the term to the maturity of its bond portfolio. This resulted 
in increased rates of return for the Fund. The specific results of the rate 
adjustment and the change in investment strategy will be reported to the 
AgriCorp board of directors. 





AgriCorp reviews its reinsurance requirements on an annual basis and 
adjusts its purchase to reflect changing risk profiles and fund balances. 
AgriCorp will continue to review its reinsurance needs on a regular basis and 
will obtain appropriate approvals from the AgriCorp board prior to the actual 
purchase of reinsurance. 


Over 20 reinsurance carriers bid for the Ontario Crop Insurance Fund 
business each year and the lowest cost providers are selected. Within the 
next year, AgriCorp will review its brokerage arrangement to determine if the 
current agreement offers the best value among reinsurance brokerage 
competitors. 











TRAVEL EXPENDITURES 


AgriCorp’s employee travel expense procedures indicate that employees should be reimbursed 
for legitimate work-related expenses and also that the most practical and economical 
arrangements for travel, meals, accommodation, hospitality and other expenses should be made. 
We reviewed the travel claims process and noted numerous examples where employees and 
the managers that approved these claims had disregarded the Corporation’s travel expense 
procedures. For example: 


* In many instances AgriCorp reimbursed expenses that were of questionable business 
legitimacy. Some examples include reimbursements for golf green fees, flower 
arrangements for staff, alcohol, gifts for staff and business contacts, umbrellas, and tickets 
for sporting and cultural events. We also noted several instances where employees 


personally benefitted by receiving promotional credits as a result of corporate purchases at 
department stores. 


Employees are required to provide the name and affiliation of each guest receiving 
hospitality at the Corporation’s expense. However, we noted numerous instances where 
employees had not done so. We also noted that hospitality, which in our view was overly 
generous and too frequent, was extended by employees to other employees at the 
taxpayers’ expense. Reimbursements were made for the meals of employees’ spouses. 


Managers were not adequately scrutinizing travel expense claims as we noted that 
employees had been reimbursed for expenses without receipts and that the receipts 
submitted were sometimes inappropriate, such as photocopied receipts and credit card 


statements rather than original receipts. Consequently, although the amounts were not 
significant, we found examples of duplicate payments. 
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Recommendation 


To ensure that employees are properly reimbursed only for work-related 
expenditures, AgriCorp should: 


¢ not reimburse employees for expenses that are of questionable business 
legitimacy; 

e develop clear guidelines outlining when the extension of hospitality at 
the Corporation’s expense is appropriate; 


e reimburse only those claims for reimbursement that are accompanied by 
proper documentation; and 


¢ ensure that managers approve only travel expense claims that comply 
with corporate procedures. 


Corporation Response 


AgriCorp will clarify its expense reimbursement policies and ensure that all 
employees are familiar with them. Management will be required to exercise 
more care in the review and approval process. 








PURCHASING 


AgriCorp’s purchasing department procedures require that goods are to be acquired using a 
competitive process or through standing agreements with suppliers. Its procedures also allow 
for the purchase of goods through petty cash or an employee’s travel expense account. 
However, these two alternatives are to be used for the purchase of goods in emergency 
situations only. 


We reviewed a sample of purchases made by the purchasing department and noted that a 
competitive process had been undertaken when required and proper documentation was on file. 
However, we also noted that employees purchased goods using travel expense claims and petty 
cash even though the goods were not urgently required and should have been acquired by the 
purchasing department. These purchases included such items as a cellular telephone, 
calculators, a fax machine, shredders, a microwave oven, a video cassette recorder, a 
television, a refrigerator, 65 shovels and professional membership fees. 


The purchasing department has controls in place to ensure that goods are obtained for the best 
price, that volume discounts are taken and that exemption from the goods and services tax 1s 
received. Consequently, the bypassing of purchasing department controls through the 
inappropriate use of travel expense claims and petty cash results in higher costs to the taxpayer. 





5 







Recommendation 


To ensure the economic purchase of goods, AgriCorp should purchase all 
goods through the purchasing department unless the use of petty cash or 
employee travel expense claims is justified. 
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Corporation Response 


AgriCorp agrees and will implement this recommendation in conjunction 
with clarification of expense policies. 








MANAGEMENT OF MOVABLEASSETS 


The majority of the Corporation’s movable assets were computer hardware and, to a lesser 
extent, furniture and fixtures. As at March 31, 2000, the cost of recorded assets was over 
$2 million. We reviewed AgriCorp’s management of these assets and concluded that the 
controls in place to account for and safeguard assets were inadequate and ineffective, as the 
following observations illustrate: 


¢ The master asset inventory list was last updated in September 1998 and did not reflect 
subsequent purchases, disposals or reassignments of movable assets. No verification of the 
accuracy and completeness of this list had been carried out. 


* No procedures were in place to ensure that assets were retrieved from employees and 
consultants that were terminated or otherwise left the Corporation. 


¢ We were informed that one field office was responsible for its own movable asset 
inventory and that its assets were not reflected in the Corporation’s master listing. 
However, we found that this field office did not maintain a list of its movable assets. 





Recommendation 





To properly control and safeguard its movable assets from loss, AgriCorp 
should: 


* continually update its asset inventory list to ensure that the list reflects 
all purchases, disposals and reassignments of movable assets; 


* perform a periodic asset verification to identify discrepancies for 
subsequent follow-up and resolution; and 


* implement a termination checklist to ensure assets are retrieved from 
departing employees and consultants. 


Corporation Response 





AgriCorp agrees with the recommendations and will implement them. The 
termination checklist has already been implemented. 
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MEASURING AND REPORTING ON 
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PROGRAM EFFECTIVENESS 


AgriCorp’s enabling legislation, the AgriCorp Act, 1996, states that its objectives are to 
administer plans of crop insurance in addition to other duties conferred upon it by legislation or 
agreements between the Ontario and the federal governments. Subsequent to its creation, 
AgriCorp signed a Memorandum of Understanding with the Ministry, issued two annual reports 
and produced three business plans, each of which described a considerably different corporate 
vision. 


Documents Containing AgriCorp Vision Statement 


[Document | Vision Statement 


Memorandum of To become a customer focused, industry driven provider of services to 
Understanding 


agri-food businesses. 
1997/98 


To be providers of innovative business solutions for Ontario's 
Business Plan agriculture and food industries. 
1997/98 


To enhance the competitiveness of Ontario’s agriculture and food 
Annual Report industry by delivering high quality agricultural insurance and consulting 
services utilizing and building on the crop insurance corporate 
infrastructure. 


To create and operate a Crown Corporation that utilizes the 
infrastructure of the crop insurance program as a base to strengthen 
the competitiveness of Ontario’s agriculture and food industry. The 
corporation is to be operated like a private sector corporation with an 
autonomous board responsible for corporate leadership: customer 
service, cost effectiveness and innovation are key success factors. 


To enhance the global competitiveness of Ontario’s agriculture and 
food industries by providing innovative risk management solutions. 


Working together to strengthen Ontario’s agri-food industry. 
















































1998/99 
Business Plan 


















1998/99 
Annual Report 


1999/2000 
Business Plan 









The most current vision statement is “working together to strengthen Ontario’s agri-food 
industry.” This statement is vague and far broader than the legislated objectives cited above. 
Also, the number of different vision statements calls into question AgriCorp’s understanding of 
its objectives. Without a clear understanding of those objectives, the Corporation will have 
difficulty developing measures to assess the achievement of its goals. 


In its 1999/2002 Strategic Plan, AgriCorp outlined key strategies for accomplishing its most 
recent corporate vision which were to: develop a new organization model; establish new 
businesses to accelerate corporate growth; increase crop insurance operating efficiency 
through an integrated information system; develop a superior work team; and increase 
organizational effectiveness through the creation of effective planning, executing and monitoring 
processes. However, these key strategies were not linked to the Corporation’s overall legislated 


objectives. 
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AgriCorp has identified the need for measuring its effectiveness in achieving corporate 
objectives but has not defined performance measures for doing so. Without a clearly stated 
corporate vision and strategies linked to its objectives, the Corporation cannot be in a position to 
measure and report on its performance to the Legislature, the Minister or its clients. 


The Corporation has carried out reviews of such areas as customer satisfaction, information 
related to financial results and call centre usage, but has not related this activity-based 
information to its corporate objectives and desired outcomes. 





Recommendation 










To ensure that it is effective in meeting its legislated objectives, AgriCorp 
should: 





° establish a clear vision statement that reflects its legislated objectives; 


¢ develop performance measures that are linked to its established vision 
and objectives; and 


* perform the assessments necessary to determine whether its operations 
are achieving its established vision and related objectives. 


Corporation Response 


The AgriCorp board of directors has initiated a review of the Corporation’s 
mandate and vision. Performance measures and assessment details will be 
established as part of the review. 














CORPORATE GOVERNANCE 


Good governance is crucial to the successful achievement of corporate objectives. As such, 
corporate governance is generally defined as the processes and structures put in place to 
ensure that a government agency is operating effectively, fulfilling its mandate and meeting its 
objectives, and is being held accountable for the prudent expenditure of public funds. The 
governance framework for AgriCorp is established by the AgriCorp Act, 1996. The 
relationship between the government and its agencies is defined in directives approved by the 
Management Board of Cabinet, which require each agency to prepare a memorandum of 
understanding between the board chair and the responsible minister outlining the roles and 
responsibilities of each party. Effective governance is a key factor in ensuring that the citizens 
of Ontario are well served by government agencies. 


AgriCorp has a volunteer board of directors comprising 16 members appointed by the 
Lieutenant Governor in Council on the recommendation of the Minister. Board members also 
make up an executive board that is responsible for AgriCorp’s overall operations. Reporting to 


the executive board and also made up of board members are a crop insurance committee and 
an audit and finance committee. 


In 1997, the board adopted a model of governance that, in theory, made it responsible for: 
overseeing the manner in which business was conducted by the Corporation; monitoring 
management to ensure that all major issues affecting the Corporation were appropriately 
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discussed; and obtaining sufficient information to allow informed decisions to be made. 
However, responses from board members to our questionnaire indicated the following 
weaknesses: 


¢ Board members as well as senior staff we talked to indicated that the governance structure 
was not working well. They cited the large size of the board, conflicts among committees 
and a lack of accountability as major barriers to good governance. Board members also 
indicated that the lines of authority and communications were unclear, including the roles 
and responsibilities of individual directors and committees. 


¢ Board members also indicated that they had not been given written explanations of their 
role and responsibilities, including the expectations, terms and conditions of their 
appointments. In addition, they indicated that the role and responsibilities of the board chair 
were not clearly defined. 


¢ The board is dependent on management for information relevant to proposed courses of 
action requiring board approval. Board members stated that they were often not provided 
with adequate information upon which to assess strategic issues or alternative courses of 
action. 


To further illustrate this concern, in February 2000, a consultant reported that “some 
information of a strategic nature, particularly relating to major information technology 
decisions, has been provided to the board in a fashion that appears to be forcing or 
manipulating a decision (insufficient time for consultation and consideration, no 
commentary, and limited business case information). This is particularly of concern with an 
executive board with little information technology exposure.” The consultant also raised 
concerns about the ability of AgriCorp’s information systems to generate timely, reliable 
and accurate information for the board to base informed decisions on. 


* One principle of good governance is that a board should be prepared to act to ensure that 
the Corporation’s objectives are met and that performance is satisfactory. The majority of 
board members stated that they acted on a timely basis when presented with clear 
evidence of a problem. However, many members felt that making decisions was difficult 
because some board members represented special interest groups rather than the interests 
of the Corporation. 


In normal situations, the board should not be involved in the day-to-day management functions 
of the Corporation. However, the board had an obligation to assume a more active role since 
concems were expressed that the Corporation did not have sufficient funds to meet its 
obligations, and the board did not have confidence that senior management could rectify the 
situation. 








Recommendation 


To improve corporate governance, AgriCorp should: 
¢ review the board and its committee structure and establish clear lines of 
communication and accountability; 


* ensure that management provides timely, sufficient and appropriate 
information for decision-making; and 
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* periodically assess the effectiveness of the board with respect to 
governance and the attainment of corporate objectives. 


Corporation Response 


The board of directors is committed to modifying its corporate governance 
structure and processes, which will include the following features: 


¢ the development of a new board/committee structure with clear lines of 
communication and accountability; 

¢ the development of processes to enhance its oversight responsibility; and 

lee periodic assessment of the board’s effectiveness. 











MINISTRY RESPONSIBILITY 


The Ministry of Agriculture, Food and Rural Affairs is responsible for AgriCorp and as such is 
required to monitor its activities to ensure that the Corporation’s mandate is being fulfilled in 
compliance with legislative and government policies. In November 1996, the Ministry developed 
a Memorandum of Understanding with AgriCorp that outlined their respective roles and 
responsibilities. These responsibilities include periodic reporting, financial arrangements and the 
appointment of a senior ministry representative to the AgriCorp board of directors as an ex- 
officio (non-voting) member. 


The financial arrangements in the Memorandum of Understanding stipulate that the records, 
systems and management practices of AgriCorp shall be kept and maintained in such a manner 
as will provide reasonable assurance that: assets are safeguarded and controlled; transactions 
are made in accordance with legislation and agreements; and financial, human and physical 
resources are managed economically and efficiently and its operations are carried out 
effectively. However, such procedures were not in place to ensure that a reasonable level of 
assurance was obtained. 


One monitoring mechanism the Ministry could employ to obtain a reasonable level of assurance 
regarding AgriCorp’s financial arrangements is the internal audit process. Pursuant to 
Management Board of Cabinet directives for the administration of agencies such as AgriCorp, 
internal audits appropriate to the needs of the agency must be performed on a periodic basis. 
However, at the time of our audit, no internal audits had ever been performed at AgriCorp. 


We have raised a number of major concerns in this report, which, once brought to its attention, 
the Ministry acted upon. However, we are concerned that such issues had to be brought to the 
Ministry’s attention in this manner. The Ministry had appointed a senior manager to AgriCorp’s 
board to provide oversight on its behalf. This representative is well-positioned to ensure that 


policies and procedures are put in place by the board to provide assurance that the Corporation 
operates as intended. 


In February 2000, the Management Board of Cabinet issued an Agency Establishment and 
Accountability Directive to provide a framework of accountability to govern the operation of 
agencies established by the Province of Ontario. In addition, since the signing of the 
Memorandum of Understanding, a new minister and deputy minister have been appointed, the 
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chair of the board has changed and the Corporation has engaged a new chief executive officer. 
Consequently, the Memorandum must be reaffirmed or revised. 








Recommendation 


To help ensure that AgriCorp is effectively fulfilling its mandate and that its 
operations are economic, efficient and in accordance with legislation, the 
Ministry should: 


° update its Memorandum of Understanding with the Corporation to include 
any new monitoring procedures and Management Board of Cabinet 
requirements; 

¢ outline specific roles and responsibilities for the ministry representative 
on the board of directors; 

¢ develop and implement procedures to more effectively monitor the 
activities of the corporation; and 

¢ implement a periodic internal audit process to provide assurance that the 
Corporation fulfills its financial arrangements stipulated in the 
Memorandum of Understanding. 





Ministry Response 


The Provincial Auditor is the auditor of AgriCorp as outlined in legislation. As 
noted in the Provincial Auditor's report, the Ministry has always acted 
promptly when concerns have been identified, including the initial comments 
of the annual financial audit of AgriCorp. Once the Provincial Auditor made 
known the audit findings, the Ministry proceeded to address these concerns 
on a priority basis. 





The Ministry agrees to review its Memorandum of Understanding to ensure 
that it complies with new monitoring procedures. Specifically, the Ministry 
will review the new Directive on Transfer Payment Accountability with the 
new board of directors and ensure that all mandatory requirements are 
implemented. It will also refer to the Best Practices Guidelines and 
incorporate reasonable best practices. The Ministry feels that an appropriate 
balance needs to be maintained that will allow the Corporation to be operated 
as an arm's length entity while upholding principles of sound administrative 
and fiscal practices that are consistent with government guidelines. 


In reviewing its Memorandum of Understanding with AgriCorp, the Ministry 
will ensure that roles and responsibilities for all board members are more 
clearly defined. This includes that of the ministry representative on the board 
of directors, who will now become a full voting member on the board. In 
reviewing these roles and responsibilities, the Ministry will refer to the 
Guidance for Directors—Governance Process for Control published by the 
Canadian Institute for Chartered Accountants. 
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The Ministry will increase the resources for agency administration and 
monitoring and they will report to the Director, Policy and Programs Branch, 
Policy and Farm Finance Division. These resources will support the ministry 
representative on the board of directors, as well as provide a point of contact 
for all agency/ministry relations. The individual will be responsible for 
ensuring the compliance of the above Memorandum of Understanding and 
any required reporting documents or procedures. In addition, the Ministry will 
continue to develop an annual performance contract with the agency. 


The Ministry will implement a periodic internal audit process to ensure that 
the Corporation fulfils its financial arrangements and will assist in the 
monitoring of the Corporation’s internal operations and procedures. 











eee 


46 fi 
Office of the Provincial Auditor 
































MINISTRY OF COMMUNITY AND 
SOCIAL SERVICES 


Child Welfare Services 
Program 





BACKGROUND 


Child welfare services in Ontario are governed by the Child and Family Services Act (CFSA) 
and regulations, as well as standards and guidelines issued by the Ministry of Community and 
Social Services pursuant to the Act and regulations. The Ministry is responsible for funding and 
setting expectations for child welfare services through legislation and regulation, as well as 
monitoring the performance of Children’s Aid Societies and taking corrective action as 
necessary. Children’s Aid Societies (Societies) have primary responsibility for providing child 
welfare services in accordance with the CFSA. Under the Act, the mandate of the Societies is 
to: 


¢ investigate and assess child abuse and neglect allegations; 


* provide guidance, counselling and other services that protect children from neglect and 
physical, sexual and emotional abuse; 


* provide temporary or permanent guardianship including residential care for children 
separated from their families; and 


¢ where appropriate, place children for adoption. 


The child welfare system in Ontario has recently undergone significant changes resulting from 
widespread concerns about child protection practices, policies and legislation. 


Child protection services are available to children under the age of 16 years. However, the 
Societies can also provide services to youth up to the age of 18 years, based on mutual consent, 
as well as supplemental extended care and maintenance support to former Crown wards up to 
the age of 21 years. 


The Ministry provides funding to 53 local Societies for the delivery of child welfare services. 
The Ministry has a network of nine regional offices which coordinate service planning and 
monitor the activities of the Societies. 

Each of the Societies is governed by an independent volunteer board of directors. Historically, 
the Ministry and local municipalities have shared funding for the Societies on an 80:20 basis 
respectively. However, since January 1, 1998, the Ministry has provided 100% of program 
funding under the government’s local services realignment initiative. 
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The Ministry estimated that the Societies provided various types of services to approximately 
154,000 children from 86,000 families over the course of the fiscal year ending March 31, 2000. 
Approximately 13,300 of these children were in residential care. Program expenditures during 
that year totalled approximately $654 million, of which $642 million was for transfer payments 
to the Societies. 


The number of children served as of the end of each of the last five years is shown in the 
following graph. 


Volume Indicators 
as of December 31 of each year indicated 
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AUDIT OBJECTIVES AND SCOPE 


The objectives of our audit were to assess whether the Ministry’s monitoring of the Societies 
was adequate to determine whether: 


children in need were receiving the appropriate care and protection, as required by 
legislation, and in an effective manner; and 


the program was being delivered with due regard for economy and efficiency. 


The scope of our audit included a review and analysis of relevant ministry files and 
administrative procedures as well as interviews with appropriate staff at the Ministry’s head 
office and three regional offices that accounted for about 50% of total program expenditures. 
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We also reviewed case files and held discussions with staff at three Societies that accounted 
for about 30% of total program expenditures. 


Prior to the commencement of our audit, we identified the audit criteria that would be used to 
address our audit objectives. These were reviewed and agreed to by senior ministry 
management. 


We substantially completed our audit examination by March 31, 2000. Our audit was performed 
in accordance with standards for assurance engagements, encompassing value for money and 
compliance, established by the Canadian Institute of Chartered Accountants, and accordingly 
included such tests and other procedures as we considered necessary in the circumstances. 


Our audit included a review of the audit reports issued by the Ministry’s Internal Audit 
Services. However, we were unable to reduce the extent of our audit as a result of this review 
because, although Internal Audit Services had reviewed individual Societies, it had not 
conducted any system-wide reviews of the Child Welfare Services program. 


OVERALL AUDIT CONCLUSIONS 


The Ministry has taken a number of steps since our last audit in 1994 to help ensure that 
children in need receive the necessary care and protection on a timely basis. However, if the 
Ministry is to be assured that children in need are being adequately protected, the Ministry must 
more effectively monitor the Societies. Specifically, the Ministry needed to take action so that: 


«the Societies conduct and adequately document the work required under the new Risk 
Assessment Model for Child Protection in Ontario in order to demonstrate that they are 
properly assessing the needs of children reported to be in need; 


- where required, the plan of service that outlines the actions needed to protect each child is 
prepared and implemented on a timely basis; 


* all instances of non-compliance with program requirements, including those related to 
serious occurrences, are identified and acted upon on a timely basis; and 


* program outcome effectiveness measures are developed and implemented. 


We also concluded that the Ministry’s new funding framework, which is to be fully implemented 
in the 2000/01 fiscal year, is a substantial improvement over the Ministry’s previous method of 
determining the Societies’ program funding. However, the Ministry will not realize the benefits 
of the new funding framework until it: 


* — ensures that the per diem funding rates for residential group home and foster care are 
based on an assessment of the nature and the reasonableness of the cost of the underlying 
services to be provided to meet the needs of the child; 


* ensures that funding for direct service costs is based on reasonable workload standards and 
salary levels that are achievable in the year to which the funding relates; 


*  reassesses whether the current practice of funding Societies’ indirect service costs based 
on a fixed percentage of net direct service costs is reflective of the actual indirect service 
costs they incur; and 
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* ensures that service data on which all funding decisions are now based are complete and 
accurate and continue to be on an ongoing basis. 


DETAILED AUDIT OBSERVATIONS 


Since our last report on this program in 1994, the child welfare system in Ontario has undergone 
significant changes resulting from widespread concerns about the capacity of the Societies to 
adequately protect children at risk. In response to inquest findings and recommendations, and a 
series of reports such as the Ontario Child Mortality Task Force Report (July 1997), Report 
of the Panel of Experts on Child Protection (March 1998) and Child Welfare 

Accountability Review (January 1997), the Ministry developed a step-by-step plan to 
strengthen the child protection system. Components of this plan include the following initiatives: 


* amendments to the Child and Family Services Act that provide stronger protection tools 
for the courts, professionals and front line staff by stating clearly that the best interests of 
children must come first; 


* — introduction of the new Risk Assessment Model for Child Protection in Ontario for 
assessing children reported to be in need of care and protection; 


¢ revitalization of foster care, including increased rates for foster parents, enhanced foster 
parent training programs and a foster care recruitment strategy; 


* introduction of a new funding framework that provides a more rational and equitable 
approach to funding the Societies based on service volumes; 


¢ development of a new information database to help track high-risk families across the 
province, which is expected to be fully implemented by March 31, 2000; and 


* adoption of outcome measures to assess the overall effectiveness of child welfare services 
that will be piloted in 2000/01. 


CASE MANAGEMENT 


ELIGIBILITY FOR PROTECTION SERVICES AND THE 
PROTECTION INVESTIGATION 


Assessing the circumstances of children reported to be at risk and determining the appropriate 
protective services required has always been one of the most important components of the 
Child Welfare Services program. However, prior to 1998, the Societies did not have a consistent 


method to assess children reported to be at risk. In fact, only half of the Societies used some 
type of structured risk assessment tool at all. 


Recognizing the need for more consistency in the assessment of children at risk and acting on 
the recommendations of several coroner’s juries investigating the deaths of children under the 
care of a Society, the Ministry developed the Risk Assessment Model for Child Protection in 
Ontario, which was implemented across the province in September 1998. This model provides a 
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structured and standardized framework for making critical decisions in child protection cases. It 
includes the following three components: 


¢ Aneligibility assessment, to determine whether the referral meets any one of the 
established eligibility criteria for protective services and the concern is of sufficient severity 
to require protective service. The outcome must be documented within 24 hours of the 
referral. 


¢ A safety assessment, which must be undertaken at the time of initial referral and first 
contact with the child (first contact must occur within 12 hours or seven days of the 
referral, depending on the circumstances), to determine the child’s immediate safety within 
the family home and whether the child should be removed immediately. This assessment 
must be documented within 24 hours of first contact. 


¢ A risk assessment, which must be documented within 30 days after initiating a protection 
investigation and at regular intervals thereafter, to assess the potential harm to the child and 
determine the type and level of ongoing intervention required, if any. 


Once the protection investigation is completed a determination is made as to whether the child 
is in need of protection and ongoing protection services. If the child is deemed to be not in need 
of protection, the protection investigation file is closed. 


We noted that the Ministry did not have a process to monitor and assess whether Societies 
were complying with the requirements of the Risk Assessment Model for Child Protection in 
Ontario. 


During the summer of 1999, the Ministry conducted a province-wide Services and Financial 
Data Review, which included a review of approximately 4,400 case files across all Societies. 
The primary objective of this review was to determine the accuracy of reported caseload data 
for current and future funding decisions (see Financial Accountability for further discussion), 
rather than for determining compliance with case management standards in general or the Risk 
Assessment Model for Child Protection in Ontario in particular. As a result, neither the 
individual reports issued to the Societies nor the province-wide summary report issued in 
January 2000 evaluated compliance with the Risk Assessment Model for Child Protection in 
Ontario or made any recommendations for improvements in that regard. 


When we prepared a summary of the files reviewed by the Ministry for the three regional 
offices we visited, we noted non-compliance with the Risk Assessment Model for Child 
Protection as follows: 


¢ for approximately 10% of the referrals, where the child was found not to be in need of 


protection as a result of the protection investigation, there was little or no documentation to 
justify closing the protection investigation file; and 


- for one region, ministry reviewers noted that half of the closed protection investigation files 
contained sufficient indications that further investigative work should have been conducted 
before the files were closed. 


As a result, the Ministry had insufficient assurance that children’s needs were adequately 
assessed and the appropriate protection services had been provided in these instances. 
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Recommendation 


To help ensure that children reported to be at risk receive the appropriate 
protection in compliance with the Risk Assessment Model for Child 
Protection in Ontario, the Ministry should: 


¢ conduct periodic file reviews to assess compliance with case 
management standards in general and the Risk Assessment Model in 
particular; and 


* identify instances of non-compliance and ensure that the necessary 
corrective actions are taken on a timely basis. 


Ministry Response 


As part of child welfare reform, the Ministry has developed a strategy to 
improve accountability of the Children’s Aid Societies in relation to the 
services that are provided. 


Under this strategy, effective 2001/02, the Ministry will conduct annual 
reviews of child protection cases using a child protection case monitoring 
tool that will be developed and piloted in 2000/01. These reviews will monitor 
Children’s Aid Society compliance with legislative and regulatory 
requirements, including compliance with case management standards and 
the Risk Assessment Model for Child Protection in Ontario, which includes 
the eligibility spectrum. 


Societies will be required to submit a plan to the regional offices to address 
any non-compliance and the Ministry will monitor implementation of this 
action plan. 





See S25] 








PROTECTIVE SERVICES AND PLANS OF SERVICE 


When a Society’s worker determines that a child is in need of protection but not in immediate 
danger, protective action must commence within 30 days of the initial referral (60 days in 
exceptional circumstances) and a plan of service must be prepared and approved by a 
supervisor within 60 days. The objective of the plan of service is to document the course of 


action that will reduce or eliminate future abuse or neglect of the child and promote the child’s 
best interests, protection and well-being. 


The Ministry does not have a process to monitor compliance with all of these requirements on a 
regular basis. 


When we prepared a summary of the files reviewed by the Ministry as part of its Services and 
Financial Data Review for children transferred to ongoing protection for the three regional 
offices we visited, we found that: 


For 16% of the files, it took more than the maximum 30 days for the investigation to be 
completed. None of these files were considered exceptional circumstances by the Ministry. 


In approximately half of these cases, the time taken to complete the investigation exceeded 
90 days. 
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¢ For 15% of the files, there was no plan of service on file. The Ministry did not ensure that 
such plans were subsequently prepared. 


Consequently, the Ministry had no assurance that all the children who needed protective action 
received it on a timely basis. 





Recommendation 


To help ensure that minimum service expectations for all child protection 
cases are met, the Ministry should monitor the Societies to make sure that: 


¢ services for children determined to be in need of protection commence 
within the required 30 days of the initial referral; and 


¢ plans of service for these children are prepared and approved by a 
supervisor within 60 days, as required. 


Ministry Response | 


In 2000/01, the Ministry will develop and pilot a child protection case 
monitoring tool for implementation in 2001/02. The Ministry will use this tool 
to monitor Children’s Aid Society compliance with legislative and regulatory 
requirements, including compliance with required timelines for 
commencement of child protection services and required timeframes for 
completion and approvals of plans of service. 





Societies will be required to submit a plan to the regional offices to address 
any non-compliance and the Ministry will monitor implementation of this 
action plan. 





CHILDREN IN RESIDENTIAL CARE 
CROWN WARD REVIEWS 


A Crown ward is a child who is a permanent ward of the Crown by order of a court. Parental 
rights and involvement in the upbringing of the child are terminated and are carried out by a 
Society. At December 31, 1999, there were approximately 5,200 Crown wards in Ontario. 


Section 66 of the Child and Family Services Act requires the Ministry to annually review the 
status of every child who is and has been a Crown ward throughout the preceding 24 months. 
Crown ward reviews examine compliance with regulatory service requirements, such as 
assessing the suitability of the child’s placement and the adequacy of the plan of care as well as 
the care provided. The results of these reviews are reported to the responsible Society. 


Where a Crown ward review finds non-compliance with a regulatory requirement, the Ministry 
issues a directive to the Society to remedy the non-compliance. The Society must comply with 
the directive, and confirm with the Ministry that it has done so, within 60 days of the directive’s 
issuance. The Ministry may also issue recommendations for less important clinical issues of 
case management or for compliance with suggested best practices. Recommendations may 
become precursors for future directives. However, the Societies are not required to act on the 
Ministry’s recommendations or confirm actions taken. 
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During 1999, 2,710 Crown wards received the legally required reviews. The results of these 
reviews are shown in the following table. 


Results of Crown Ward Reviews, 1999 


Full compliance 
Directives and/or recommendations issued 











Some non-compliance but no directives and/or 
recommendations issued 





Source: Ministry of Community and Social Services 


We reviewed a sample of cases for which directives or recommendations were issued and 
noted the following: 


¢ In approximately 15% of the cases of regulatory non-compliance, the Ministry issued a 
recommendation instead of the required directive. As a result, the Society was not 
compelled to remedy the non-compliance and the Ministry was not required to follow up on 
its concerns. 

¢ For approximately 20% of the files where a directive was issued, we found no evidence 
that the Society acted on the directive and confirmed its actions to the Ministry within the 
60 day requirement. 

















Recommendation 


To help ensure that services provided to Crown wards are appropriate and 
in compliance with regulatory requirements, the Ministry should: 


¢ issue directives in all instances where program regulations have not 
been complied with; and 


* monitor Children’s Aid Societies to ensure they act on all directives and 
confirm compliance with the directive to the Ministry within 60 days, as 
required. 


Ministry Response 


The Ministry requires that directives be issued in all instances of Children’s 

Aid Society non-compliance with program regulations identified in a Crown 
ward review, except where the issuance of a directive cannot be enforced or 
may be deemed inappropriate. 





The Children ’s Aid Society is required to develop and submit a plan to the 
Ministry to address non-compliance. Ministry regional offices are required to 


review and sign off on all Society plans and monitor for follow-up action, if 
necessary. 
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In an effort to enhance this monitoring process, the Ministry is developing an 
automated “bring forward” system to promote prompt ministry follow-up of 
society action on non-compliance within the required 60 days. 





NON-CROWN WARD REVIEWS 


Approximately 8,100 children were non-Crown wards on December 31, 1999. Although child 
welfare program requirements apply equally to both Crown wards and non-Crown wards, the 
Child and Family Services Act does not specifically require the Ministry to monitor program 
delivery for non-Crown wards. As a result, the Ministry historically has not reviewed non- 
Crown ward files. 


However, we noted that the 1997 Child Welfare Accountability Review recommended that 
the best practices and lessons learned from Crown ward reviews should be used as a model for 
reviewing the progress of all children in care, regardless of their legal status. 


Our review of a sample of non-Crown ward files identified a number of program delivery 
deficiencies that also demonstrated the need for the Ministry to review non-Crown ward files. 
For example: 


¢ for 30% of the files we reviewed in one regional office, we found no evidence that the 
protection worker visited the child within seven days of the original placement in a foster 
home, as required by regulation; and 


¢ for 20% of the files, the required school report was not on file, as required by regulation. 


We also noted that an internal ministry review found that approximately 10% of the placements 
of non-Crown wards were either not appropriate to their care or the placements were not 
assessed. 


The Ministry recognizes the need to review non-Crown ward files. We understand that, at the 
time of our audit, the Ministry was in the process of piloting a non-Crown ward review process 
at selected Societies throughout the province. 














Recommendation 


To help ensure that protective services provided to non-Crown wards are 
appropriate and in compliance with program requirements, the Ministry 
should: 





¢ implementa regular review process for non-Crown ward files; and 


* ensure instances of non-compliance with program requirements are 
communicated to the Children’s Aid Society and, where required, acted 
upon on a timely basis. 





Ministry Response 


The Child Welfare Review Unit of the Ministry has a comprehensive 
information system and review mechanism in place for Crown wards that has 
recently been expanded to include a sample of other children in Children’s 
Aid Society care. 
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An implementation plan for enhancing case monitoring of non-Crown wards 
was initiated in October 1999. Based on the findings and further consultation 
with key stakeholders, the Ministry will conduct annual reviews ona sample 
of non-Crown wards, effective August 2000. As in the ministry reviews of 
Crown wards, directives will be issued for non-compliance of program 
regulations and Children’s Aid Societies will be required to respond within 60 
days. All Societies’ responses will then be reviewed and signed off by the 
ministry regional offices and monitored for follow-up action, if necessary. 














EXTENDED CARE AND MAINTENANCE AGREEMENTS 


Crown ward status terminates at the age of 18 years. However, former Crown wards between 
the ages of 18 and 21 years are eligible for extended care and maintenance, including ongoing 
financial support (currently $663 per month), provided they are prepared to enter into a written 
agreement with their Society to work towards specific goals during that time period. Specific 
goals typically include such things as going to school, learning a trade or seeking employment. 


In December 1999, there were approximately 1,140 extended care and maintenance 
agreements requiring annual ministry expenditures totalling $6.9 million. Ministry instructions for 
the completion of extended care and maintenance agreements provide that the frequency and 
means of contact between the youth and the youth’s case worker may vary significantly 
depending on the needs and circumstances of the individual youth. For example, according to 
the Ministry’s instructions, a youth needing more support might contract with the worker for 
contact once per month. 


Our review of a number of extended care and maintenance agreements found that a majority 
of them required a monthly meeting between the youth and the case worker. However, our 
review also found that the Societies were not adequately monitoring these youths, as required 
by their agreement, to ensure that the agreed-upon goals were being achieved and the 
conditions of funding were met. Specifically: 


* For about one third of the files we reviewed, we found no evidence that the monthly 
meeting occurred as required by the agreement. 


* For one half of the files we reviewed, we found no documentation on file to demonstrate 
that the youth’s personal goals were being met. 


As a result of the above, we question whether the program was adequately monitored to assess 
whether or not the program’s objectives were being met. 








Recommendation 


To help ensure the successful transition of Crown wards to independent 
living, the Ministry should ensure that: 


Children’s Aid Societies regularly monitor youths who have extended 


care and maintenance agreements, as required by their agreements, and 
assess compliance with their personal goals; and 
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e where the requirements of the agreements are not met, corrective action 
is taken by the Societies on a timely basis. 









Ministry Response 


The extended care and maintenance agreements must be reviewed and 
renewed by the Society annually. Children’s Aid Societies have the authority 
to approve extended care and maintenance agreements for a second and 
third year without ministry approval of the renewed agreement. 


The Ministry will develop a plan to monitor adherence to the extended care 
and maintenance agreements. 








LICENSING OF CHILDREN’S RESIDENCES 


Licensing provisions for children’s residences are established by legislation and regulation and 
are intended to ensure that minimum acceptable standards of care are provided to children in 
residential care. Ministry licensing inspections include the completion of a licensing checklist 
covering such things as a review of the premises, the services provided and the adequacy of 
the organization’s policies and procedures. 


Children’s residences and foster care operators must apply for a licence renewal annually, prior 
to the expiry date of the current licence. Provided that the applicant has completed and 
submitted an application for renewal, a licence past its due date is deemed to continue until the 
request for renewal is granted or denied. 


Our review of a sample of licensing files noted the following concerns: 


* For approximately 80% of the files reviewed, the Ministry conducted the required licence 
review and issued the renewal licence after the expiry of the previous licence. The average 
delay between licence expiry and renewal was 63 days, with some as long as 200 days. 
The reasons for the delays were not documented. 


- At one regional office, almost half the files we reviewed lacked the required documentation 
to support the issuing of the licence. 


© For 23% of the files reviewed, the renewal application was either not received at all or was 
received after the expiry of the previous licence. As a result, these operators were 
unlicensed for an average of approximately one month and, in one case, 88 days. 


Consequently, there was a lack of timely assurance that a number of children’s residences 
continued to provide acceptable standards of care. 





Recommendation 


To encourage residential care operators to provide minimum acceptable 
standards of care to children, the Ministry should conduct and document 
licensing inspections and renew licences on a more timely basis. | 
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Ministry Response 






In September 1999, the Ministry introduced automated business tools to 
enhance local, timely management of licence renewals and to support 
province-wide monitoring of this activity. Furthermore, in the fall of 1999 the 
Ministry designated specific staff in its regional offices to supervise and 
support the licensing function. 


In an effort to further enhance the effectiveness of the ministry licensing 
process, the timing of the licensing of Children’s Aid Society foster care will 
be synchronized to coincide with the review cycle of the ministry Child 
Welfare Review Unit in 2000/01. 











SERIOUS OCCURRENCES 


At the time of the audit, the Ministry required all Societies to report incidents such as serious 
injuries, assaults or other physical abuse of children in care within 24 hours of the occurrence. 
When such reports are received, regional offices are required to document the particulars in an 
initial notification report. In addition, the Societies must submit a written follow-up report to the 
regional office detailing the corrective actions to be taken within five working days of the 
incident. The Ministry is to review the report and follow up, if necessary. 


Our review of a sample of serious occurrence files found that these reporting requirements 
were often not being followed. For example: 


¢« for almost half of the serious occurrence files we reviewed, we found that an initial 
notification report had not been prepared; 


¢ for almost three quarters of the written follow-up reports that we reviewed, there was no 
evidence that ministry staff had reviewed them or evaluated the appropriateness of the 
corrective actions taken; and 


* one regional office could not locate any of the 1998 serious occurrence files for two of its 
Societies. 


As a result, the Ministry could not demonstrate that all serious occurrences had been 
appropriately dealt with. 











Recommendation 







To help safeguard children in care and gain assurance that all serious 
occurrences are appropriately dealt with, the Ministry should take the 
necessary steps to ensure that: 


° an initial notification report is prepared for every serious occurrence 
reported; and 


* written follow-up reports from the Societies are evaluated to determine 
whether appropriate actions have been taken. 
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Ministry Response 






The Ministry is committed to ongoing efforts to guide improvements to the 
reporting and documentation of all serious occurrences by all service 
providers, including Children’s Aid Societies, and to assist staff in follow-up 
actions. 


The Ministry has recentiy revised the standards and guidelines for the 
reporting of all serious occurrences across services, including those 
provided by the Children’s Aid Society. These revised standards and 
guidelines require that the regional office document the review and follow up 
with a service provider for each serious occurrence to assess whether that 
service provider has taken appropriate corrective action. 


These revised materials will be forwarded to the regional offices for 
distribution to service providers by October 2000. 


PROGRAM EFFECTIVENESS MEASURES 


The systematic collection of case data and determination of outcomes provides a basis for 
evaluating program effectiveness as well as identifying best practices. However, at the time of 
our audit, the Ministry was not systematically collecting outcome data and had not assessed 
outcome measures for the Child Welfare Services program. As a result, the Ministry could not 
assess the effectiveness of the child welfare system as a whole in such areas as the quality of 
care provided, the progress of children in care and the rate of recurrence of maltreatment. This 
was also noted in an October 1999 research study funded by the Ministry. 


The Ministry is aware of the need to collect outcome data and to assess outcome measures 
and, in the fall of 1998, adopted an outcome measurement framework developed by a national 
working group. The Ministry has selected three of a potential ten outcome indicators 
(recurrence, placement rates and number of moves in care) for priority pilot testing in 2000/01. 


We will follow up on the Ministry’s efforts to develop and implement outcome indicators to gain 
assurance that the child welfare program is operating effectively as soon as practicable. 


FINANCIAL ACCOUNTABILITY 
FUNDING FRAMEWORK 


Prior to the 1998/99 fiscal year, funding to the Societies was primarily based on annual budget 
requests, which in turn were largely based on historical funding patterns. As noted in our 1997 
report on Transfer Payment Agency Accountability and Governance, this funding mechanism 
did not relate an agency’s funding approval to an assessment of the value of the underlying 


services to be provided. 


As a result of such concerns and in order to promote greater funding equity among the 53 

Societies, in December 1998, the Ministry announced a new funding framework that would 
provide a more rational and equitable approach to funding the Societies. This new funding 
framework is primarily based on an assessment of the type and volume of services provided, 
and is being phased in over a three-year period. Although the new funding framework will be 
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fully implemented in 2000/01, ministry policy ensured that funding in 1999/2000 was not less 
than in the previous year. 


Funding under the new framework consists of the following three main components: 


* approximately half of a Society’s funding is for residential care, which is based on the 
number of children in residential group home and foster care at per diem rates developed 
by the Ministry; 


* approximately one quarter of a Society’s funding is for direct service costs for staff 
salaries, which are based on caseloads and ministry-determined workload benchmarks and 
salary ranges; and 


* approximately one quarter of a Society’s funding is for indirect costs, which are determined 
as a percentage of the first two funding components. 


We consider the Ministry’s new funding framework to be a significant improvement over the 
previous funding mechanism. 


While we fully agree with the Ministry’s direction in this area, our review of the implementation 
of the new funding framework for the 1999/2000 fiscal year noted a number of areas that 
needed to be addressed if the full benefits of the new funding mechanism are to be achieved. 


RESIDENTIAL CARE 


Group Home Per Diems 


The Ministry has established the per diem for all categories of the group home type of 
residential care at $142.18 per day, based on the average cost of group care purchased from 
outside paid institutions as reported in a 1997 survey of all Societies. 


However, according to the same survey results, the actual cost of care in group homes 
operated by the Societies (which often provide homes to the more difficult children) and the 
cost of special rate agreements for children with special needs were significantly higher, 
averaging approximately $190 per day. The actual costs ranged from a low of $72 per day to a 
high of $581 per day. 


We also noted that the Ministry did not conduct its own assessment of what reasonable costs 
should be for each category of residential care. As a result, the Ministry did not know whether 
the average per diem rate paid as a result of the 1997 survey was reasonable or adequate to 
provide the required level of care at particular Societies. 


Foster Care Per Diems 


There are essentially three categories of the foster type of care: regular foster care; specialized 
foster care for children with developmental, emotional or medical needs: and treatment foster 
care for children who require behaviour modification treatment and care. Foster care may be 
purchased or provided in society-operated foster care homes. 


The Ministry has established per diems for each category of care as shown in the following 
table. 
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Foster Care Per Diem Rates, by Type 










$32.20 per day 
Specialized $49.76 per day 
$67.64 per day 


Source: Ministry of Community and Social Services 


We noted that the regular and specialized foster care rates are based on a review of rates for 
such care in other jurisdictions and the Federal Guidelines for Child Support, while the 
treatment rate is based on the average cost of purchased treatment foster care reported in the 
1997 survey of all Societies. 


However, although the Ministry determined through a 1998/99 survey that the average cost of 
treatment foster care provided in society-operated homes, which generally provide homes to the 
more difficult children, averaged over $100 per day, they are nevertheless funded at the rate of 
$67.64 per day. 


DIRECT SERVICE COSTS 


Direct service costs consist primarily of the salary and benefits costs of the front line workers 
who provide and coordinate both residential and non-residential care and protection of children. 
Funding for direct service costs was based on the type and volume of direct services provided, 
as well as workload benchmarks and salary ranges determined under the funding framework. 


Our review of this aspect of the funding framework noted the following concerns: 


© Workload benchmarks were determined by the Ministry based on a review of similar 
benchmarks in other jurisdictions as well as focus groups. The Ministry did not conduct its 
own review and assessment of the underlying work to be performed to determine the 
reasonableness of these standards. 


¢ Final funding for direct service costs was based on 94% of the maximum salary of the 
Societies’ staff and workers in other sectors performing similar functions. This salary often 
did not reflect actual salaries paid and was likely not achievable by many Societies in the 
short term. 


Consequently, funding was not reflective of the actual cost incurred and levels of care 
provided. 


INDIRECT SERVICE COSTS 


The Ministry’s funding framework provides for different types of indirect costs as a percentage 
of total costs net of revenues raised, as shown in the following table. 
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Indirect Service Costs by Type as a Percentage of Total Costs 


Central Administration 


* includes such items as human resources, legal services, 
information systems and property maintenance 


Program Administration 


* includes all other types of administrative costs directly 
attributable to a program 


Support Services 


¢ includes professional fees, children’s personal needs and 
health related expenses 














Source: Ministry of Community and Social Services 


The central administration funding benchmark was based on the average actual central 
administration costs relative to net direct service expenditures of all Societies as determined in 
the Ministry’s 1997 survey. The benchmark is also consistent with the Ministry’s current policy 
to limit central administration costs for all types of transfer payment agencies to 11% for the 
1999/2000 fiscal year. 


However, staff at the regional offices and the Societies that we visited expressed concern with 
how central administration funding was determined. Primarily, they felt that higher direct 
service cost agencies, which often utilize more expensive forms of residential care, did not 
necessarily incur higher central administration costs yet received increased funding nonetheless. 
In addition, larger and consequently higher-cost Societies could be expected to realize some 
economies of scale. These concerns bring into question the merits of funding indirect costs for 
all Societies at the same fixed percentage of net total costs. 


In that regard, we noted that: 


* central administration costs for the agencies we reviewed ranged from 4.6% to 13% of 
their net direct service costs; and 


* program administration costs ranged from 6% to 13.1% of their net direct service costs. 


As a result, funding for indirect service costs was not reflective of the actual costs incurred by 
the Societies. 


REVENUE BENCHMARKS 


The Societies are expected to raise 6.7% of their annual gross expenditures themselves. These 
revenues normally include such items as parental contributions, tax rebates, charitable donations 
and any interest earned. 


Although we were advised that the 6.7% revenue benchmark was based on a review of actual 
receipts by the Societies and their potential for increased revenues, most of the Societies we 
reviewed could not meet that goal. In fact, while actual revenues for all Societies averaged 
approximately 4.8%, some were as low as 3%. 
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Recommendation 


To ensure that funding provided to Societies under the new funding 
framework is appropriate for them to deliver the required services, the 
| Ministry should ensure that: 


e residential care per diems are based on an assessment of what 
reasonable costs should be for each type and category of cave: 


¢ direct service cost funding is based on a ministry review and assessment | 
of the underlying work to be performed; 

* funding for indirect service costs is reasonable and appropriate for each 
Society; and 

e revenue benchmarks are reasonable and attainable. 


Ministry Response 


The Ministry is currently conducting an independent review of the child 
welfare funding framework benchmarks/design elements for: 


* group care; 
e indirect service costs, including legal services, travel costs and other 
program support costs; and 


e @6revenue. 


This review is to identify options and recommendations that reflect careful 
analysis of financial, service management and service delivery implications, 
and Children’s Aid Society policy/best practice considerations. This review 
will be completed in September 2000. 


In addition, the Ministry will conduct a comprehensive review of the child 
welfare funding framework model design and accompanying benchmarks in 
three-year cycles, commencing in 2001/02. Adjustments to the framework will 
be considered, where necessary, upon completion of each review cycle. 








SERVICEAND FINANCIAL DATAREVIEW 


Since the Ministry’s new funding framework for the Societies is primarily based on caseload 
data, it is essential that caseload data reported by the Societies are complete and accurate to 
ensure the integrity of the funding process. 


As a result, during the summer of 1999, the Ministry conducted an extensive one-time review 
of caseload data at all 53 Societies. The objectives of this review were to determine whether 

reported caseload data used for the 1999/2000 fiscal year funding allocations for each Society 
were complete and accurate and, where necessary, provide recommendations for 


improvements. 
The review noted that two thirds of all Societies had unreliable non-residential caseload data. 


The Ministry estimated that for Societies that had the most unreliable data, non-residential . 
caseloads were over-reported by an average of 9%. This resulted in excess ministry funding of 
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approximately $6.7 million for these Societies alone. For example, many Societies reported 
cases as requiring ongoing protection when in fact they should have been closed. 


At the time of our audit in February 2000, the Ministry had established a pilot review process 
for the Societies. However, the necessary policies and procedures for such reviews by the 
regional offices, such as their frequency and sample size determinations, had not yet been 
finalized. 





Recommendation 


In order to maintain the integrity of the Ministry’s new funding framework, 
which is primarily based on caseload volume data, the Ministry should: 


¢ establish ongoing requirements for and conduct regular Children’s Aid 
Society caseload data reviews; and 


¢ adopt appropriate policies and procedures for such reviews. 


Ministry Response 





The child welfare funding framework relies on the provision of accurate and 
reliable service and financial information by Children’s Aid Societies to 
determine system resource requirements and individual society funding 
allocations. 


Building on the results of the 1999 Service and Financial Data Review 
process, the Ministry has developed a draft monitoring and control protocol 
for regional office staff to assess the integrity and reliability of Children’s Aid 
Society service and financial data in relation to the funding framework. 


This monitoring protocol includes: increased regional office visits to 
Children’s Aid Societies and random case file reviews; detailed reviews of 
agency quarterly reporting data to ensure that it provides a sound basis upon 
which to determine funding allocations; and enhanced in-year service volume 
and expenditure forecasts. This monitoring protocol will be finalized for 
ongoing implementation in 2000/01. 


Regional offices completed reviews in November 1999 and February 2000. 














QUARTERLY REPORTING 


The Societies are required to submit quarterly reports comparing actual to budgeted expenditure 
and caseload data. The first three quarterly reports are due 30 days after the end of the quarter 
and the fourth quarter report is due 45 days after the year-end. As part of the reporting 
process, the Societies are required to identify and explain any significant variances and provide 
an appropriate action plan. Timely receipt and analysis of the quarterly report are particularly 
important now that the Societies’ funding is based on actual caseload data. 


Our review of quarterly reports at the three regional offices we visited identified a number of 
concerns as follows: 
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* Three quarters of the quarterly reports we reviewed were received after their due dates. In 


addition, many of the reports lacked the required explanations of significant variances and 
action plans, which limited their usefulness to the Ministry. 


* Variance analysis included in the quarterly reports generally lacked sufficient detail to 
permit their effective review and analysis. For example, expenditure and caseload volume 
data were not presented for each category of care within a specific type of care. 


* We found no evidence on file that ministry staff analyzed and followed up on significant 
variances identified. 


As a result, we found evidence that significant in-year variances were not identified, analyzed 
and followed up for corrective action by the Ministry on a timely basis. 






Recommendation 


In order to ensure the timely identification, analysis and follow-up of 
significant in-year variances in expenditure and caseload data, the Ministry 
should: 

















e obtain the required quarterly report variance analysis on a timely basis 
and in sufficient detail, including the necessary explanations and 
corrective actions; and 


¢ clearly document its process for reviewing variances and, if necessary, 
- approving the appropriate corrective actions. 


Ministry Response 


The Ministry agrees that Children’s Aid Society quarterly reports must be 
timely and provide sufficient detail to explain and address in-year variances. 


Subsequent to the periods reviewed by the Provincial Auditor, the Ministry 
enhanced the quarterly report format to capture more detailed service and 
financial data from the Societies. This new report format includes variance 
analysis on all expenditures by ministry project codes. The Ministry continues 
to refine this new quarterly reporting process based on ministry and 
Children’s Aid Society feedback and experience. 


The new quarterly review process also now provides 20 days for regional 
office analysis, and consultation with the Societies prior to uploading of the 
reports to the ministry Service Management Information System. 








ANNUALPROGRAMEXPENDITURE RECONCILIATION 


The purpose of the Annual Program Expenditure Reconciliation (APER) is to reconcile a 
Society’s eligible expenditures with the funding provided by the Ministry in order to identify any 
surpluses or ineligible expenditures. APERs must be received together with an audited financial 
statement no later than four months after each fiscal year. The Ministry 1s to review and 
approve the APER within 12 months of the end of the fiscal year to which it relates and is to 


recover any surplus funding within 24 months. 
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We reviewed a sample of APERs for the 1997/98 and 1998/99 fiscal years and concluded that 
the reconciliation process was ineffective for the reasons noted below: 


Approximately 75% of the APERs we reviewed were received after their due dates. On 
average, these APERs were received six months late and, in one case, a required APER 
for 1997 was not received at all. 


For approximately 40% of the APERs we reviewed, we found that the accompanying 
audited financial statement lacked sufficient detail to identify ineligible expenditures or 
determine the accuracy of the reported surplus or deficit. 


The Ministry often did not review and approve APERs on a timely basis. Many were 
reviewed about 20 months after the end of the fiscal year, while one was not reviewed at 
all. 


We also noted that, at the time of our audit, the Ministry had not developed a reconciliation 
format to be used with the new funding framework, which is to be fully implemented in the 
2000/01 fiscal year. 








feeer its Management Support Branch. 








Recommendation 


To ensure that it identifies and recovers ineligible expenditures and surplus 
funding, if any, the Ministry should: 





e receive, review and approve all Annual Program Expenditure 
Reconciliations on a timely basis; and 


¢ ensure that Annual Program Expenditure Reconciliations and 
accompanying audited financial statements contain sufficiently detailed 
and comparable information to identify ineligible expenditures or surplus 
funding. 


Ministry Response 


For the 1999/2000 fiscal year, the Ministry replaced the use of the Annual 
Program Expenditure Reconciliation (APER) form for the Child Welfare 
Program with a new report format that provides significant details to identify 
eligible expenditures and surplus funding at Children’s Aid Societies. 


In order to improve the effectiveness of the year-end reconciliation process, 
the Ministry developed an APER—Best Practices Tool-kit to assist regional 
offices with the timely completion and review of these reports. This process 
will expedite the year-end reconciliation to allow the Ministry to recover 
surpluses on a more timely basis. 


A policy regarding mandatory reporting requirements and sanctions for 
agency non-compliance was provided to ministry regional offices in October 
1999 and outlines an incremental four-step process to acquire overdue 
information from agencies, including Children’s Aid Societies. In addition, the 
Ministry monitors all year-end reconciliation reports, including Child Welfare, 
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OTHER MATTER 


INFORMATION SYSTEMS 


At the time of our audit, the Ministry did not have a province-wide information system in place 
that could facilitate the collection of data for use in tracking and assessing child welfare 
services. However, recognizing the need for a comprehensive information system, in 1997, the 
Ministry commenced the child welfare information system project, which consisted of two 
distinct components: 


* The Fast Track Information System that enables all Societies to track high-risk families and 
alert other Societies to a family’s previous involvement with other child welfare authorities. 
Development of the system has been completed and it was fully implemented by all 
Societies by March 2000. 


A comprehensive information system that will collect and summarize a wide range of data 
from all Societies. This system is intended to assist in case management and monitoring, 
and the reporting of aggregate information for management purposes. We were informed 
that the development of this component is approximately three to four years behind the 
original expected completion date of March 2001. 


Recommendation 


In order to facilitate the collection and analysis of performance data 
necessary to effectively administer the Child Welfare Services program, the 
Ministry should expedite the development of its proposed management 
information system. 


Ministry Response 


The Fast Track Information System is now in place in all Children’s Aid 
Societies. This is the first phase of a two-phase plan to implement a 
Comprehensive Child Welfare Information System. 


The Ministry is currently in the process of determining its next steps in the 
development of a comprehensive Child Welfare Information System. 
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MINISTRY OF CONSUMER AND 
COMMERCIAL RELATIONS 


Project to Automate the 


Land Registration System 
(POLARIS) 





BACKGROUND 


In 1980, the Ministry established POLARIS (Province of Ontario Land Registration System), a 
project to automate the province’s land registration system. Documents and plans affecting land 
in Ontario had been maintained under either the Registry System or the Land Titles System. 
Until the implementation of the POLARIS project, both of these systems were exclusively 
maintained in paper and microfilm form at land registry offices operated by the province. The 
POLARIS project involves the gradual conversion of all properties in the Registry System to 
the Land Titles System and the automation of records to permit the searching and registration 
of real property documents through the use of computers. Prior to the commencement of this 
project, approximately 70% of properties in Ontario were registered under the Registry System. 


In 1991, Teranet Land Information Services Inc. (Teranet)—a corporation owned jointly 
between the province and a private sector company—assumed from the Ministry financial and 
contractual responsibility for the implementation and operation of POLARIS. At the time, the 
Ministry had converted approximately 250,000 properties to the POLARIS electronic format. 
Teranet’s services also include providing users with online remote access to the POLARIS 
database and allowing users to electronically register documents that affect land title. 


As part of the new venture, the Ministry provided Teranet with a Licence Agreement, which 
transferred ownership of POLARIS to Teranet, including all software, hardware, licences and 
trademarks associated with POLARIS. This transfer did not include original land registration 
documents, the information contained in these documents, and survey and parcel mapping data. 


Teranet, which holds a conditional exclusive licence for providing automated land registration 
services, receives fees collected by the Ministry for registration and ancillary transactions that 
the Ministry processes using POLARIS. The Ministry receives from Teranet a royalty of 25% 
for registration-related revenue and 5% for ancillary non-search services and certain other 
services. 


The Ministry owns 40% of voting shares in the corporation, plus other special shares that entitle 
the Ministry to 50% of dividends and distributions declared by Teranet. The province and the 
private sector partner were required to provide $29 million each to Teranet for their equity in 
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the corporation. In addition, when the corporation was established, the Ministry received 
preferred shares (non-voting, non-participating), redeemable for $30 million, for its contribution 
of work already completed on the project and for POLARIS. 


The Shareholder Agreement provides that the Ministry and the private sector partner are each 
entitled to nominate four of Teranet’s 11 directors and jointly nominate another three. In 
addition, the Shareholder Agreement requires that certain matters be approved by 75% of the 
directors present at a meeting and restricts the distribution of profits. As well, Ministry approval 
is generally required should the private sector partner change ownership. 


As of March 31, 2000, approximately 2.5 million of the estimated 4.3 million properties in the 
province had been fully converted to electronic format. Since 1991, the Ministry has, out of the 
Consolidated Revenue Fund, paid Teranet $235 million from revenue obtained for automated 
land registration and ancillary services, of which Teranet has paid back $45 million in royalties 
to the Ministry. The majority of Teranet’s revenues, along with funds obtained through 
financing, have been used for ongoing project costs incurred to date and the operation and 
maintenance of the automated system. 


REVIEW OBJECTIVE AND SCOPE 


In the course of conducting a financial controls review of the Ministry, we identified significant 
issues and concerns regarding the status of the POLARIS project and the Ministry’s 
arrangement with Teranet. Our objective was to determine whether the Ministry had 
adequately assessed the impact of its arrangement with Teranet on the continued operation of 
the land registration system and on the province’s investment in the project. 


The scope of our review, which was substantially completed in April 2000, included interviews 
with appropriate staff and an examination and analysis of documentation available to the 
Ministry. Our review was conducted primarily at the Ministry’s Registration Division. We did 
not rely on the work of the Ministry’s Internal Audit Services because they had not conducted 
any recent work that was relevant to our review. 


Our review was conducted in accordance with the standards for assurance engagements 
established by the Canadian Institute of Chartered Accountants and accordingly included such 
tests and other procedures as we considered necessary in the circumstances. 


OVERALL REVIEW CONCLUSIONS 


We identified a number of significant concerns with the Ministry’s arrangements with Teranet 
and the status of its project to automate the province’s land registration system. In 1991, the 
POLARIS project had an original cost estimate of $275 million and an anticipated 1999 
completion date. In April 1999, Teranet provided the Ministry with an estimate of over $700 
million to complete the project and a projected completion date of 2010. According toa 
consultant hired by the Ministry, due to significant uncertainties in the assumptions used by 





69 
Special Report: Accountability and Value for Money 


2 


Teranet, the project could cost over $1 billion. Consequently, the Ministry’s risk, costs and 
benefits with respect to the project have changed considerably. 


In September 1999, Teranet pledged most of its significant assets as security for a $280-million 
bond offering, primarily to repay debts and to fund further implementation of the project. This 
financial restructuring by Teranet could increase the cost to the Ministry in the event that the 
Ministry chooses to terminate its agreement with Teranet and assume operation and control of 
the POLARIS project. 


We plan to follow up on the project to automate the province’s land registration system in 
future, since the Ministry had not yet decided on an appropriate course of action. 


DETAILED REVIEW OBSERVATIONS 
PROJECT COMPLETION DATE AND COSTS 


In 1991, an Implementation and Operation Agreement was established that required Teranet to 
achieve certain benchmarks in connection with the automation of the land registration system. 
Specifically, Teranet was required to perform all work necessary to automate and convert 
properties in Ontario in accordance with the progress schedule and timetable stated in the 
agreement. 


The original Implementation and Operation Agreement called for automating all properties by 
November 1999. In December 1993, the agreement was amended to require fulfillment by 
November 2000. In January 1998, the Ministry agreed to a contractual extension and a further 
revision of the project’s completion date to March 2004. At the time of our review, the Ministry 
was evaluating a forecast from Teranet, dated April 1999, that indicated a further extension to 
2010 was required. The Ministry advised us that they had not agreed to further changes to the 
contractual obligations of Teranet for completing the project. 


In December 1999, Teranet terminated its contract with its principal subcontractor, who had 
been converting land registration information to electronic format. This resulted in a reduction in 
the number of properties being automated from about 23,000 per month during 1999 to about 
12,000 for February 2000. Although ministry staff informed us that they believe Teranet will 
meet its existing contractual benchmarks for the year ending March 31, 2001, the information 
we obtained indicated that Teranet will not be able to automate a sufficient number of 
properties to meet such benchmarks for the year ending March 31, 2002, or for future years. 


In addition, we noted that cost estimates to complete the project had substantially increased. In 
1991, the initial estimate to complete the project was $275 million. However, in June 1998, ina 
report to the company’s bankers, Teranet’s auditors stated that projected costs were expected 
to be $560 million. As of April 1999, Teranet estimated that the total costs for the project would 
be more than $700 million, and this was based on Teranet’s assumption that certain cost-saving 
methods could be implemented. 


In September 1999, the Ministry engaged a consultant to review Teranet’s POLARIS project, 
which included providing advice on the Ministry’s financial risk with respect to the project. In its 
November 1999 report, the consultant noted the following: 
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* — Teranet’s estimate of over $700 million in total project costs is one of many scenarios. 
There are significant uncertainties in the assumptions that were utilized to develop the 


estimate. Other, less favourable scenarios estimate total project costs in excess of 
$1 billion. 


* The cost to convert the remaining 1.8 million properties using existing workflows and 
processes exceeds the anticipated revenues from conversions. 


*  Teranet should reassess its estimation procedures for determining project conversion costs 
and timetables, and the Ministry should independently review Teranet’s procedures and cost 
estimates. 


The consultant also made several recommendations for improving Teranet’s accountability 
process to ensure that relevant operational and financial matters are brought to the attention of 
key ministry officials on a timely basis. 


We found evidence that corroborated the risks identified by the consultant. 


In addition, we noted that the Ministry had initially anticipated that transferring the POLARIS 
project to Teranet would maintain government revenue. However, for fiscal 1994/95, the 
Ministry’s revenues, net of expenditures, from land registration transactions were $38 million; 
by fiscal 1999/2000, the Ministry’s net revenues, including royalties received from Teranet, had 
declined to about $13 million. In addition, Teranet had not declared any dividends to its 
shareholders since its inception, and, as of March 31, 2000, it had an accumulated deficit of 
$44 million. 


OTHER COSTS 


In the event that Teranet does not perform in accordance with the terms of its Implementation 
and Operation Agreement, such as inadequate progress made on automating land registration 
information, the Ministry has the option of issuing a notice of termination of the agreement to 
Teranet. In this event, the matter is referred to an arbitrator to determine if termination is 
appropriate under the circumstances. If the arbitrator decides it is appropriate, the arbitrator 
can then determine fair and equitable compensation for either party in the circumstances. 


We noted that unless an adequate contingency plan is in place, termination of the agreement 
could result in increased risk for the ongoing, uninterrupted operation of the province’s land 
registration system. However, the Ministry asserted that should its agreement with Teranet be 
terminated, there were adequate safeguards in place to ensure the continued operation of the 
province’s land registration system. For instance, upon termination, the agreement permits the 
Ministry to assume operation and control of POLARIS in an orderly manner. 


We also noted that a recent financing restructuring by Teranet could increase costs to the 
Ministry should it choose to assume operation and control of the POLARIS system. In 
September 1999, Teranet issued $280 million in bonds, which mature September 8, 2009. 
Teranet used the net proceeds from the sale of the bonds primarily to repay existing bank loans 
and to finance its future implementation costs for automating land registration information. The 
province acquired $30 million of these bonds in exchange for the preferred shares of equal 
value that it acquired when Teranet was established. 
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As security for the bonds it issued, Teranet pledged most of its significant assets, including 
monetary assets held by the company, revenues and receivables, computer software, equipment 
and agreements. This includes the Licence and the Implementation and Operation agreements 
between Teranet and the province. As a result, if the Ministry were required to terminate its 
agreement with Teranet and assume operation and control of the project, it would have to 
address Teranet’s obligation to the bondholders. In addition, the Ministry may have to 
compensate Teranet for all or part of the costs it has incurred. For instance, since 1991, Teranet 
has substantially invested in automating land registration documents and further developing the 
POLARIS system. As of March 31, 2000, Teranet reported these costs to be over $300 million. 


FUTURE FOLLOW-UP 


As indicated earlier, the Ministry advised us that they had not agreed to further changes to the 
contractual obligations of Teranet for completing the POLARIS project. The Ministry also 
advised us that it was in the process of assessing its options for automating the land registration 
system, implementing improvements to its accountability process with Teranet and deciding on 
an appropriate course of action. We plan to follow up on the Ministry’s progress in due course. 
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MINISTRY OF CORRECTIONAL SERVICES 


Institutional Services and 
Young Offender 
Operations 





BACKGROUND 


The Ministry of Correctional Services supervises the detention and release of inmates, parolees, 
probationers and young offenders. Its objectives are to ensure the protection and security of 
society and to motivate offenders towards positive personal change. To do this, it creates for 
the inmates a social environment in which they may achieve changes in attitude through 
training, treatment and services designed to afford them the opportunities for successful 
personal and social adjustment in the community. 


The Ministry is authorized to incarcerate offenders under the federal Prisons and 
Reformatories Act and Young Offenders Act, and the Ontario Ministry of Correctional 
Services Act. 


Institutional Services and Young Offender Operations (Institutional Services) is the ministry 
activity responsible for the operation of Ontario’s correctional institutions, including jails, 
detention centres and correction centres. These institutions provide custody for adult offenders 
sentenced to terms of up to two years less a day and for accused persons on remand awaiting 
trial. They also provide custody for young offenders between 16 and 17 years of age. 


For the 1999/2000 fiscal year, Institutional Services had operating expenditures of . 
approximately $463 million and about 6,200 staff. On a daily basis, there were approximately 
7,400 adult and 700 young offenders in 47 correctional institutions. 


AUDIT OBJECTIVES AND SCOPE 


Our audit objectives were to assess whether the Ministry had adequate procedures and 
systems in place to: 


* ensure institutional resources were managed with due regard for economy and efficiency; 


* ensure that institutional services and programs were delivered in accordance with 
legislative and ministry requirements; and 
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* measure and report on the effectiveness of the services and programs delivered by 
Institutional Services. 


The audit included visits to the Ministry’s corporate offices, six correctional institutions operated 
by the Ministry and one young offender facility operated by the private sector. In addition, we 
sent out questionnaires to all the correctional institutions and analyzed responses from the 
superintendents. Prior to the commencement of our audit, we identified criteria that would be 
used to address our audit objectives. These were reviewed and accepted by senior 
management of the Ministry. 


Our audit was conducted in accordance with the standards for assurance engagements, 
encompassing value for money and compliance, established by the Institute of Chartered 
Accountants, and accordingly included such tests and other procedures as we considered 
necessary in the circumstances. 


We reviewed relevant work performed by the Ministry’s Internal Audit Services Branch. It 
was useful in reducing the extent of our audit work relating to security measures for the 
correctional institutions. Our fieldwork was substantially completed by March 2000. 


OVERALL AUDIT CONCLUSIONS 


In our 1993 audit of Institutional Services, we concluded that the Ministry could realize 
significant savings through the modernization or replacement of existing correctional institutions. 
In 1996, the Ministry initiated an infrastructure renewal project with approved capital costs of 
over $270 million. When the institutions under the renewal project became fully operational, the 
Ministry expected that it would be able to significantly reduce the operating costs of 
incarcerating adult inmates—making Ontario one of the low cost provinces in Canada. 
However, we noted that in the process of implementing the renewal project, the Ministry did not 
properly assess the viability of alternative delivery options to ensure best value to the taxpayers. 


Overall, we concluded that the Ministry’s systems and procedures were not adequate to ensure 
institutional resources were managed with due regard for economy and efficiency, nor to 
ensure services and programs were delivered in accordance with legislative and ministry 
requirements. Regarding the infrastructure renewal project: 


* The Ministry’s decision to finance and construct two 1,200-bed correctional institutions that 
cost $180 million was not supported by a sound business case assessing the risks, costs and 
benefits of all feasible alternatives. 


- A proper cost-benefit analysis was not done for the building of a new cooking facility within 
a correctional institution under expansion to provide prepared food to a number of 
correctional institutions. At the time of our audit, the cost to build the facility had increased 
from $5 million to $9.5 million. 


* The Ministry had yet to realize any of the anticipated savings of $8.5 million from 
retrofitting its correctional institutions over a year before. 
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In addition, despite a decline in average inmate count in recent years, the operating 
expenditures for Institutional Services between 1995/96 and 1999/2000 had increased 19% 
from $388 million to $463 million. Specifically, we noted that: 


* The Ministry had not effectively utilized its community programs, which are designed to 
provide non-violent offenders with opportunities for successful reintegration into the 
community. In particular, we noted that the number of offenders in the Temporary Absence 
Program had declined from 25,000 to 4,000 between 1991/92 and 1998/99. The under- 
utilization of community programs resulted in the Ministry foregoing significant potential 
savings of as much as $50 million a year. 


* The Ministry was not adequately monitoring staff attendance and overtime at its 
correctional facilities. The average number of sick days for correctional officers increased 
38% from 12 days in 1995 to 16 days in 1998. Overtime expenditures increased 48% from 
, 04 $11.1 million in 1996/97 to $16.5 million in 1998/99. 


* Correctional programs were offered to adult inmates without properly assessing the 
correctional needs of individual offenders. 


* The Ministry did not ensure that its security measures were adhered to and that timely, 
corrective action was taken in cases of non-compliance. About 60% of its 47 correctional 
institutions had security non-compliance problems that had not been rectified for up to two 
years. 


We also concluded that the Ministry needed to measure and report on its effectiveness in 
protecting society and motivating offenders towards positive personal change. 





DETAILED AUDIT OBSERVATIONS 
DUE REGARD FOR ECONOMY AND EFFICIENCY 


According to ministry data, the average daily inmate count (including both adult and young 
offenders) peaked in the 1996/97 fiscal year. Since then, the average count has dropped 6% 
from 8,600 to 8,100. However, operating expenditures for Institutional Services between 
1995/96 to 1999/2000 have increased 19% from $388 million to $463 million. 
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Average Daily Inmate Count and Operating Expenditures 
1995/96 to 1999/2000 
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A review of the latest information provided by Statistics Canada showed that the average daily 
cost per adult inmate for all provinces except Ontario was $108 in the 1997/98 fiscal year. 


Ontario’s per diem cost of $136 was the second highest of all the provinces, as shown on the 
following chart. 


Average Per Diem Cost per Adult Inmate 
by Province, 1997/98 
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Canada-wide cost information for young offenders was not available from Statistics Canada. 
According to the Ministry, the annual cost of incarcerating a young offender in Ontario 
amounted to over $95,000 compared to $50,000 for an adult offender. 


In our audit of Institutional Services in 1993, we reported that the high per diem cost in Ontario 
was due to factors relating to the inefficiency of certain smaller and older correctional facilities, 
a lower utilization of alternative community programs than other provinces, and high staffing 
costs. 


To provide guidelines for ministries in implementing the government policy of pursuing better 
alternatives in delivering services, the Management Board of Cabinet issued an Alternative 
Service Delivery (ASD) Framework in 1996. All ministries were directed to consider 
alternative service delivery options, including privatization, of all their programs and activities to 
limit expenditures and provide better services. 


Under the ASD Framework, the choice of delivery method should be supported by a sound 
business case, which includes the assessment of: 


* the costs and benefits of all feasible alternatives, including consideration of all relevant 
factors; 


* the risks associated with the proposed delivery option, including risks allocated between the 
province and service provider, as well as those transferable to the private sector; and 


* the protection of public interest. 


As part of our audit, we assessed whether Management Board’s guidelines in exploring 
alternative service delivery of existing programs and new initiatives were being followed. As 
indicated in the following sections of our report, we found that the guidelines, highlighting 
prudent business practices, generally were not followed. Specifically: 


¢ Business cases to support the Ministry’s choices of delivery methods were either not done 
or were prepared without proper consideration of all relevant factors. 


¢ In assessing the viability of private sector involvement, the level of risks that could be 
transferred to the private sector had not been adequately considered. 


¢ Proper consideration had not been given to balancing private sector objectives with the 
interests of the public. 


ADULT INFRASTRUCTURE RENEWAL PROJECT 


In 1993, we found that Ontario’s smaller adult institutions had higher per diem costs than 
average. These institutions tended to be older with inefficient design that resulted in some staff 
to inmate ratios 2.5 times higher than the average of larger institutions. We observed that | 
improving the efficiency of the smaller institutions could require considerable long-term capital 
investment. We concluded that significant savings could be realized through modernization or 
replacement of existing facilities. 


In 1996, the Ministry engaged two external consulting firms to verify the merits of proceeding 
with the modernization and replacement of its adult correctional facilities. In the fall of 1996, the 
Ministry initiated the Adult Infrastructure Renewal Project with approved total costs of over 

$270 million for various capital projects to reduce operating costs and increase the efficiency of adult 
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correctional facilities. We reviewed the following capital projects which had been started or 
completed at the time of our audit: 


* Construction of two new 1,200-bed facilities in Penetanguishene and Lindsay—approved 
total costs amounted to $87 million for each of the two facilities. The construction contracts 
were tendered and awarded to contractors in 1998 at $83 million and $79 million 
respectively. At the completion of our audit in March 2000, the Penetanguishene project 
was near two-thirds completion and the Lindsay project was at one-quarter completion. 


* Expansion of an existing facility in Milton to increase the capacity from 600 beds to 1,500 
beds—approved total costs amounted to $90 million. The construction contract was 
tendered and awarded to a contractor at $79 million in 1998. The Milton expansion project 
was near three-quarters completion. 


¢ Retrofit of three existing detention centres (including Metro Toronto East, Metro Toronto 
West and Hamilton-Wentworth) with a combined capacity of 1,400 beds—approved total 
costs amounted to $8 million. The contracts were tendered and awarded in 1998 at a total 
contract cost of about $7 million. The project in Hamilton-Wentworth was close to 
completion, while the two projects in Metro Toronto had been completed in March 1999. 


The above facilities with a total of 5,300 beds would accommodate almost two thirds of 
Ontario’s adult inmates. When the new facilities become fully operational, the Ministry expects 
that it will be able to significantly reduce the operating costs of incarcerating adult inmates. For 
example, the per diem cost of the two new facilities was projected to be between $56 and $60, 
which would be among the lowest in Canada. 


NEW FACILITIES 


The two new correctional facilities in Penetanguishene and Lindsay were designed to be 
virtually identical. This would enable more meaningful comparison of capital costs, operating 
costs and other performance measures. As part of exploring alternative service delivery 
options, in August 1996, Management Board directed the Ministry to prepare a draft request for 
qualification (RFQ) for the two new facilities. The RFQ was to obtain information regarding 
private sector qualification and feedback on the financing and ownership of the facilities. 


The Ministry informed us that the RFQ was never prepared due to time constraints, although it 
did meet with the interested private sector consortia. Ultimately, both of the facilities were 
financed, constructed and owned by the province without private sector participation. 


According to ministry officials, private financing and ownership was not viable in 1998. 
However, at the completion of our audit, the Ministry indicated that it was exploring the 
possibility of having the private sector operate at least one of the facilities. We were informed 
that the Ministry was preparing an RFQ to gauge private sector interest in operating the 
Penetanguishene facility, which was scheduled to be completed in the fall of 2000. 


The Ministry indicated that the reasons for not having private sector financing and ownership of 
the facilities included the following: 


¢ Inthe meetings, the private sector consortia indicated they would be willing to provide 
the financing to win the contract. However, they were more interested in obtaining the 
operating component of the package. 
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* With provincial financing and ownership, the cost of financing would be at the province’s 
interest rate, which was 15 to 40 basis points lower than that of the private sector. 


* Ifa facility were to be owned and operated by the private sector, the province’s ability to 
regain possession and/or ownership in the case of inadequate performance could be 
problematic. 


* The release of a request for qualification (RFQ) or a request for proposal (RFP) for private 
sector proponents to finance, design/build, own and/or operate the facility could result in 
significant time delays caused by union grievances. 


Despite the reasons provided above, we noted that the Ministry’s decision regarding the choice 
of financing and ownership was not based on a sound business case to ensure “best value for 
the tax dollar” as recommended under the ASD Framework issued by Management Board. 
Specifically, there was no cost/benefit analysis done using the recommended approach (that is, 
net present value) to choose the option with the highest net benefit. As well, there was no 
consideration of risks to be transferred or managed with respect to each option. 


For the private sector to finance, build and own correctional facilities would require significant 
capital investment. As well, the private sector would have to assume the risk of cost overruns, 
which could significantly affect the return on the investment to their shareholders if the risk was 
not properly managed. It is therefore understandable that private sector operators would prefer 
to obtain only the operating contract in running the facilities. However, in other jurisdictions 
such as the U.S. and U.K., a major investment of private sector capital is common practice 
when private operators compete for the operating contracts of correctional facilities. 


When the marginally higher interest rate of 0.15% to 0.4% was mentioned as one reason for 
having the province finance, construct and own the facilities, there was no consideration of 
transferring the risk of cost overruns to the private sector. Based on our review of change 
orders and revised cost projections provided by the Ministry, we noted that the Penetanguishene 
project, at about two-thirds completion and with a construction contract price of $83 million, will 
cost $92 million to complete. This would be $5 million more than the total cost of $87 million 
approved by Management Board. 


The Ministry’s concern regarding the possibility of inadequate performance of the private 
sector owner/operator is a valid concern. However, without the long-term commitment of 
private sector equity, the risk of inadequate performance or non-performance could be even 
higher. Since there is no risk of losing any investment of capital, a private sector operator would 
not have as much to lose, other than the operating contract, in case of inadequate performance. 
In addition, to increase profit for the operator, regular building maintenance could be deferred if 
the operator did not also have an interest in ownership. This could result in significant capital 
expenditures to repair infrastructure in the long run when operation is returned to the province. 
Private operators would have more incentive to ensure proper upkeep of facilities that they also 
own. 


The Ministry was directed to explore alternative service delivery options including privatization 
as far back as 1996. We were concerned that a properly prepared business case was not 
available to address all significant issues, such as the cost/benefit of private sector involvement, 
repossession in case of inadequate performance, union grievances and protection of the public 


interest. 
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EXPANSION PROJECT 


The expansion project would increase the existing capacity of the institution in Milton from 600 
beds to 1,500 beds, making it the largest correctional facility in Canada. 


Our examination of change orders and revised projected cost figures provided by the Ministry 
revealed the following: 


¢ At about three-quarters completion, the expansion project with a construction contract price 
of $79 million will require about $93 million to complete. This would be $3 million more than 
the total cost of $90 million approved by Management Board. 


* Much of the cost increase was due to underestimating the costs of building a cooking 
facility (called cook-chill) when the expansion project was originally approved. 


Cook-Chill Facility 


Cook-chill utilizes a food processing system that prepares food to a “just done” state followed 
by rapid chilling. The facility was to provide prepared food to ten correctional institutions at a 
capital cost of $5 million. 


Originally, the facility was to be financed and operated by the private sector. However, the 
Ministry subsequently decided against private sector involvement because there were no 
experienced private sector operators, labour costs would be higher should unpaid inmate labour 
not be used, and any problems with the quality of the food could result in inmate unrest in all the 
institutions served by the cook-chill facility. As well, the private sector could choose not to use 
inmate labour, resulting in inmates being deprived of a useful vocational training experience. 


At the time of our audit, we noted that the estimated costs of the cook-chill facility had 
increased from $5 million to $9.5 million. Instead of serving the needs of ten institutions, the 
food prepared would only serve six. In addition, the equipment to retrofit three existing 
institutions alone would cost $3.9 million, when equipment costs for these institutions were 
originally estimated to be less than $100,000. 


Our review of the cook-chill project showed that the cost-benefit analysis was not properly 
done before making the decision to build the facility. Specifically: 


* The existing institutions to be served already have food costs as low as that projected for 
cook-chill, about $1.80 per meal. However, the cook-chill facility could supply only about 
60% of food items. Certain types of fresh food would have to be purchased locally at 
additional cost. 


* The capacity required to serve the needs of the six institutions was 16,000 meals daily, 
seven days a week. However, the daily production capacity was limited to 15,000 meals 
because the water chilling mechanism could operate only 8 to 12 hours per day. Cold 
storage space availability also limited production capacity. 


¢ The original plan was for five days of production per week. Staffing costs would increase 
significantly if production were to be increased to seven days per week. In addition, there 
was no consideration of the additional costs required to purchase food from alternative 


sources should transporting food from the cook-chill facility become impossible due to bad 
weather. 
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At the completion of our audit, the Ministry had not decided whether it would reduce the 
number of institutions to be served, cut down the number of hot meals or supplement the food 
requirement from other sources. Also, Ministry officials indicated that they were in the process 
of preparing a RFQ/RFP to invite private sector operators to bid for an operating contract, as 
they were doubtful that the Ministry would have sufficient internal expertise to run the facility. 


With a properly prepared business case, the Ministry would have been able to more properly 
assess needs, and address risks and other logistical requirements as an integral part of its 
planning process. 


RETROFIT PROJECTS 


The retrofit projects at the three detention centres were initiated in 1996, with tendered contract 
costs totalling about $7 million. At the time of our audit, the project in Hamilton-Wentworth had 
not yet been finished while the two projects in Metro Toronto had been completed for over a 
year. Our examination of retrofit projects was confined to the two completed projects. 





The scope of retrofit work for these two projects included installing electric locks, control 
stations, closed circuit television monitoring, and security and extended emergency systems. 
With the new technology in security and monitoring, the Ministry originally anticipated a total 
cost saving of $8.5 million per year due to the reduced staff level required. Our examination of 
the two projects indicated that, although the retrofits had been completed for almost a year, 
neither of the two institutions had realized any of the anticipated savings. 


¢ In the case of the Metro Toronto East Detention Centre, the technological improvements 
were installed but had yet to be utilized at the time of our audit. 


¢ In the case of the Metro Toronto West Detention Centre, the institution had started to use 
the new monitoring technology and security equipment but the staffing level had not been 
reduced. 


The Ministry informed us at the completion of our audit that the equipment would be used at 
both facilities and that staffing reductions had been announced. However, ministry officials 
indicated that the savings would be significantly less than originally anticipated as the staffing 
reductions originally planned were found to be overstated, based on a subsequent review 
performed by an external consultant. 








Recommendation 


When evaluating alternative services delivery options, the Ministry should 
prepare sound business cases, including needs analysis and requirement 
definitions, to ensure that the option chosen will result in best value to the 
taxpayers. At the minimum, business cases should include an assessment of: 


e the costs and benefits of all feasible alternatives; 


¢ the kind of risks and the level of risks that can be transferred from the 
province to private sector partners as well as how such risks should be 


managed; and 
* the proper balance between the objectives of the private sector and the 
interests of the public. 
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To ensure realization of anticipated benefits, the Ministry should establish 
procedures for the proper planning and implementation of the chosen 
service delivery option. 


Ministry Response 


The Ministry reviewed and analyzed information concerning private sector 
financing, ownership and operation of correctional facilities. This included 
risk assessment and transfer, review of studies comparing private and public 
sector financing, ownership and operation, as well as a review of other 
jurisdictions that have opted for private sector financing and ownership. 


¢ In February 2000, the Ministry established an Alternative Services Delivery 
(ASD) Unit to manage the implementation of ASD directions approved by 
Cabinet. This unit is to operate within the parameters of the Management 
Board ASD Framework and the Guide to Implementing a Change in 
Service Delivery. 


¢ In August 2000, a request for qualification (RFQ) was issued to solicit 
expressions of interest by private or non-profit service providers to 
operate the correctional centre in Penetanguishene. The information 
obtained from these submissions and the subsequent proposals 
submitted by qualified bidders will assist the Ministry in its cost/benefit 
analysis of this ASD initiative. 


With respect to the retrofit projects, in February 2000 the Ministry gave 
OPSEU notice, in accordance with the collective agreement, that staffing 
reductions would occur at the Metro Toronto West and East Detention 
Centres. These staffing reductions were implemented at the Metro Toronto 
West Detention Centre in May 2000 and at the Metro Toronto East Detention 
Centre in July 2000. 











PRIVATIZATION OF YOUNG OFFENDER FACILITY 


In 1997, the Ministry initiated a pilot project, known as Project Turnaround, which introduced a 
more disciplined approach to the incarceration of young offenders. A Request for proposal was 
issued in February 1997 and the pilot project was awarded to the successful private bidder in 
April 1997. The project, with a capacity to accommodate 32 young offenders in Barrie, was 
turned over to a private operator in July 1997 for three years. 


The three-year contract was for $8.3 million. At the end of the contract in July 2000, the pilot 
project was to be evaluated in terms of its impact on young offender recidivism, academic 
achievement, employment success and other factors. 


Our review of the contract and payments made to the contractor showed that: 


* An additional $400,000 was paid to the contractor beyond the contract price for security 
custody services. Ministry officials informed us that the amount was for additional staffing 
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not anticipated in the original contract. Our examination of the contract did not reveal any 
provision for payments beyond the contract price. 


* The Ministry was not verifying invoiced amounts against the contract, It had been 
overpaying the contractor by $24,000 per year for aftercare services until we brought the 
Overpayment to its attention. 


We noted at the time of our audit that while the pilot contract had yet to be completed and the 
evaluation carried out, the Ministry had already announced its decision to expand the use of the 
strict discipline program being run by private operators. 


Recommendation 


To ensure proper control over payments to private service providers, the 
Ministry should more closely monitor the operations and billings of private 
sector partners that are providing alternative services to the province. 


To better ensure future outsourcing provides value for money, the Ministry 
| should complete its evaluation of the effectiveness of the pilot project 
before deciding whether more young offender facilities should be turned 
over to private sector operators. 





| Ministry Response 


The Ministry will undertake to more closely monitor all contracts with private 
sector partners. 


In mid-September, the Ministry will receive the final evaluation report on the 
effectiveness of Project Turnaround; preliminary reports have indicated 
positive results. On the basis of these evaluations, the Ministry will be 

_extending the operation of Project Turnaround and establishing similar young 
offender facilities in partnership with the private sector. 











COMMUNITY PROGRAMS 


To provide opportunities for the successful reintegration of inmates into the community, 
Canadian provinces have established community programs, such as the Temporary Absence 
Program or the Electronic Monitoring Program, as alternatives to incarceration. Through these 
community programs, superintendents of correctional institutions have the discretion of allowing 
inmates convicted of non-violent offences to serve all or part of their sentences in the 
community, under terms and conditions imposed by the institutions. 


These community programs contribute to the protection of society by enabling offenders to 
maintain community and family relationships and responsibilities, and to attend continued 
education, employment and other rehabilitation programs. 


Many superintendents indicated that community programs such as the Temporary Absence 
Program were more cost effective than incarceration and should be utilized more often. 
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According to ministry data, the number of temporary absences granted has decreased from 
25,000 to 4,000 between 1991/92 and 1998/99. In particular: 


* temporary absences for employment have decreased from 3,500 to about 300; and 


* temporary absences for academic/vocational training have decreased from 360 to 13. 


Number of Temporary Absences 
1991/92 to 1998/99 


30,000 5 


25,000 + 


20,000 5 


15,000 + 


10,000 4 
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Source: Ministry of Correctional Services 


Our review of the latest Statistics Canada information indicated that, in the 1997/98 fiscal year, 
Ontario had on average placed fewer than 8% of the offenders in its institutions in community 
programs. In contrast, Alberta and Manitoba had each placed 20% and Quebec had placed 
45% of sentenced offenders in community programs. Alberta, the only province to provide us 
with outcome statistics, reported that 99% of offenders in 1997/98 successfully completed the 
Temporary Absence Program without revocation. 


The majority of inmates were admitted to Ontario’s institutions for property and other offences 
not related to crimes of violence. For the last eight years, the selection criteria for temporary 
absences continued to be confined to low-risk, non-violent offenders. 


Our examination revealed that Ontario’s success rate with the Temporary Absence Program 
over the eight years remained unchanged at about 97%, with the failures attributed mainly to 
technical violations such as missing a curfew. Ministry staff indicated there was not one case of 
an offender reported to have committed a serious crime while on temporary absence. 


In addition to providing opportunities for the rehabilitation and successful reintegration of 
inmates into the community, community programs could also help the Ministry in managing its 
resources more cost effectively. For example, if Ontario had used its community programs to 
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the same extent as Alberta, it would have, on average, over 1,000 fewer adult inmates each day 
in its institutions. Given that it costs about $50,000 a year to provide accommodation for an adult 
inmate and over $95,000 a year for a young offender, better utilization of community programs 
could provide significant operational savings of as much as $50 million a year. Alternatively, it 
could enable the province to avoid the major costs of building and operating one new facility. 



















Recommendation 





To afford non-violent inmates better opportunities for successful 
reintegration into the community and to reduce institutional expenditures, 
the Ministry should make more effective use of community programs. 


Ministry Response 


Public safety is the Ministry’s top priority. When selected non-violent 
offenders are identified as being appropriate for participation in community 
programs, the Ministry will pursue this course of action. Under the new 
performance framework, adult institutions are required to deliver results- 
oriented programming and services within the institution to assist inmates 

| with their successful personal and social adjustments. 


In the Ontario Budget 2000, the addition of 165 new probation and parole 

officers was announced as part of the investment in safe communities and 

| the new $18 million strict discipline model for community corrections. These 
staff will provide more intensive and frequent monitoring of offenders and 

_assist in restricting the movements of offenders serving sentences in the 

community. 





STAFFING 
ATTENDANCE 


For security reasons, correctional officers who are absent due to sickness or other reasons 
must be replaced immediately. In many cases, this requires paying the substitute officers 
overtime at one-and-a-half times the hourly rate. 


Our examination of the latest available data on short-term sick leave of classified correctional 
officers indicated that the average number of sick days increased by 38% between 1995 and 
1998. 
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Average Number of Sick Days 
1995 to 1998 


Average Days 
12.97 
13.76 









Source: Based on data from the Ministry of Correctional Services 


Since most classified correctional officers worked 12-hour shifts, the average of 16 sick days 
could be equivalent to 24 days for staff working on 8-hour shifts. 


Data on short-term sick leave in 1999 were not available at the time of our audit. However, our 
discussion with superintendents and our review of overtime pay indicated that the situation was 
getting worse. 


To address attendance concerns, the Ministry implemented an Attendance Enhancement 
Program in January 1996 and replaced it with the Attendance Support Program in October 
1997. When we examined the attendance-monitoring effort at five of the correctional 
institutions, we noted that: 


* only one of the five institutions maintained relatively current attendance monitoring records; 
and 


¢ even for that one institution, 30% of its staff with sick days over the monitoring threshold 
were not monitored in accordance with the requirements of the Attendance Support 
Program. 


Our discussions with superintendents indicated that the poor attendance of correctional officers 
was caused by low staff morale, due to the uncertainty of their employment status since 1996. 
The superintendents maintained that excessive use of sick leave was a result of some staff 
abusing the system. 


While poor staff morale and ineffective design of the Attendance Support Program were cited 
as reasons for the excessive use of sick days, we believe that the lack of proper monitoring by 
management of staff attendance at the institutions also contributed to the problem. 


Recommendation 


The Ministry should strengthen its efforts in monitoring sick leave and, 
where warranted, take appropriate corrective action to deal with any 
problems. 


Ministry Response 


The Ministry agrees attendance is a problem and has a number of initiatives 
under way to address this issue: 
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* The Ministry is currently reviewing attendance records of individuals with 

| excessive absenteeism and considering appropriate action, up to and 

| including release. 

| The Human Resources Branch is establishing a new unit for the case 

_ management of attendance. Recruitment is nearing completion for the 
Staffing of this unit. 


© The Ministry is currently assessing the tracking and reporting features 
__ available through the newly implemented Workforce Information Network 
__ system, as well as providing input into the development of new modules, 
such as occupational health and safety, to enable the most efficient 
collection and reporting of attendance data. 


* OPSEU has agreed to work with the Ministry on a joint working group to 
deal with issues of staff morale and attendance. 


















OVERTIME 


Our review of the Ministry’s attendance management system indicated that total overtime 
hours worked by both classified (full-time permanent) and unclassified (part-time contract) 
correctional officers decreased by over 40% between 1995 and 1999. 


This declining trend however was not consistent with payroll records which showed that 
overtime pay to these officers has increased by 48% between 1997 and 1999, from $11.1 
million to over $16.5 million. According to payroll administration staff, the institutions had not 
been reporting all the overtime hours worked by their staff in the attendance management 
information system. 


During our visits to individual correctional institutions, we noted that: 


¢ Overtime hours were often assigned to classified officers at one-and-a-half times pay when 
assigning the hours to unclassified contract staff would eliminate paying overtime under 
those circumstances. 


¢ Overtime hours assigned to individual officers were not tracked to ensure that overtime 
hours were deployed in a cost-effective manner. 


Individual superintendents agreed that overtime could be better managed with closer monitoring 
and that more use of unclassified staff would eliminate or reduce overtime hours. However, the 
superintendents informed us that they did not use more unclassified staff to reduce overtime 
use as these officers sometimes were unwilling to work more hours. Also, for security and 
safety reasons, they were not allowed to perform certain responsibilities (escorting prisoners, 
for example) per local collective agreements. 


Nonetheless, all the institutions we visited had contract officers available who had not worked 
their full number of regular hours. 
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Recommendation 







To improve staff attendance and reduce costs, the Ministry should ensure 
that: 


¢ actual overtime worked is properly recorded in the attendance 
management system; 
e overtime is better tracked and monitored; and 


¢ staff with regular hours available are used before overtime hours are 
assigned. 


Ministry Response 


The Ministry is proceeding to implement an automated shift scheduling 
administration system to improve the proper recording of attendance, as well 

as to better track, monitor and assign overtime. 
ce 








COMPLIANCE WITH LEGISLATIVE AND MINISTRY 
REQUIREMENTS 


STAFF TRAINING 


To ensure cost-effective operations and the proper protection of staff and inmates in its 
correctional institutions, the Ministry has established training requirements for correctional 
officers. As well as providing correctional staff with initial training when they commence their 
employment, the Ministry also requires institutions to provide updated training for safety 
courses, such as cardiopulmonary resuscitation and first aid, at least every two years. Other 
courses are to be updated over a five-year period. 


In response to our survey, over 60% of superintendents indicated that the existing training 
required of correctional officers was not adequate to meet the operational needs of their 
institutions. When we reviewed the training provided to correctional officers, we noted that 
none of the institutions we visited had complied with the training requirements established by 
the Ministry. Specifically: 


¢ Ministry policy required correctional institutions to maintain current staff training records 
with a chronological list of the training received to date and the annual training required. We 
noted that training records at institutions we visited were either incomplete or inadequate. 


¢ Due to the suspension of the advanced correctional study training requirements during the 
last four years, over 80% of the correctional officers had not received the training required 
to keep their skills up to date. 


Without proper training records of staff, the Ministry was not in a position to establish an 
adequate training program. Without reliable records, it might be wasting resources in providing 
training to staff who had already received the necessary training. Alternatively, staff who do 
not have the necessary training might jeopardize the safety of other staff and inmates. 
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Management indicated that the Ministry was aware of this deficiency in staff training. It had 
initiated some measures to address its training needs but had been unable to fully implement 
them due to resource constraints. 



























Recommendation 


To establish training programs that help to better protect staff and inmates, 
the Ministry should: 


* provide up-to-date training to all correctional staff: and 
* maintain current staff training records. 


| Ministry Response 


The Ministry is making a significant investment in training to sustain and 
develop programs for staff: 


¢ The 2000/01 Business Plan allocated $5 million in new funding for staff 
training. 

¢ The training policy has been revised and ongoing training for correctional 
officers and managers will occur on a continual basis. 


¢ The principles established for compressed workweek schedule 
_ negotiations include a written commitment to correctional staff for a 
specific number of training hours for officers, depending on their 
specialized training needs. 

* With the recent introduction of the Workforce Information Network the 
Ministry will take advantage of the training portion of that system for 
tracking training. | 

¢ The request for qualification issued in August 2000 to solicit expressions 

of interest by private and non-profit service providers to operate the 
correctional centre in Penetanguishene included the requirement that the 
staff and management of all operators meet the Ministry’s training and 
experience requirements. 











CORRECTIONAL PROGRAMS 


Consistent with its mandate to protect society by motivating offenders towards positive personal 
change, the Ministry provides various programs to meet the correctional needs of inmates. We 
examined whether ministry policy and procedures were followed in a cost-effective manner. 


ASSESSMENT OF NEEDS 


Ministry officials indicated that inmates on remand in detention centres were assumed to have 
no correctional needs until they were convicted. In addition, over 70% of inmates were 
sentenced to Ontario correctional institutions for less than three months. Superintendents often 
expressed doubt whether it would be possible for correctional programs to affect positive 
behavioural change in light of the short stay of most offenders in their correctional institutions. 
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We examined the files of inmates who were committed to the institutions for a minimum of six 
months to see whether their correctional needs were properly identified. Our examination 
indicated that: 


¢ Initial plans of care were prepared in all cases to assess the correctional needs of young 
offenders. However, the needs assessment performed was not consistent between the two 
institutions we visited. One of the institutions provided specific details in the plans of care 
while the other did not. For both of the institutions, over half of the required plans of care 
were not completed within 30 days as required by ministry policy. 


¢ Only one of the three correctional centres for adults carried out a formal assessment of the 
correctional needs of offenders with program recommendations for meeting those needs. 
The others offered various programs without assessing whether they met the correctional 
needs of individual offenders. At these other institutions, offenders were left to identify 
their own correctional needs and to volunteer participation in the programs they wished to 
attend. 


DELIVERY OF PROGRAMS 


For the one correctional centre that did recommend programs based on individual assessment 
of inmates’ correctional needs, we noted that: 


¢ 10% of the inmates declined to participate in the recommended correctional programs; and 


¢ 27% of the correctional program recommendations for meeting the needs of inmates 
were not met as the recommended programs were not available. 


PROGRAM EVALUATION 


To allow for better allocation of resources to meet the correctional needs of offenders, the 
Ministry has scheduled program evaluations at intervals of one to three years. 


Our audit indicated that none of the institutions we visited had complied with the program 
evaluation requirements. Without program evaluations to assess whether the programs offered 
were relevant to the needs of offenders, resources devoted to such programs might not be cost 
effective in protecting the society and in motivating offenders towards positive personal change. 


Recommendation 


To better meet its mandate of protecting society and motivating offenders 
towards positive personal change, the Ministry should ensure that: 


¢ the correctional needs of offenders are properly assessed and 
addressed through the provision of appropriate programs; and 


the effectiveness of correctional programs is evaluated in a timely 
manner. 
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Ministry Response 















The Ministry agrees that properly assessing offenders, providing them with 
| appropriate programs and evaluating program effectiveness contributes to 
public safety. 


Through the recently established Program Effectiveness Unit, a number of 
business processes are being redesigned and/or formalized to ensure that the 
evaluation of program effectiveness is a core component of both new and 
_established correctional programs. 


The Ministry is also establishing outcome-based performance measures to 
_assess the effectiveness of programming for all sentenced offenders to 
determine success in lowering re-offending rates. 











PROTECTION AND SECURITY REQUIREMENTS 


To ensure that proper security measures are in place for the protection of society, staff and 
inmates, the Ministry has established policies and procedures to prevent escapes, suicides and 
other critical incidents from occurring in its institutions. However, we noted that, in practice, 
such security policies and procedures were often not properly complied with. 


The Ministry’s internal audit performed security reviews on all provincial correctional 
institutions on a two-year cycle. Non-compliance with security policies and procedures were 
reported to individual superintendents with recommendations for corrective action. Our review 
of the security reviews in the two years before our current audit showed that all 47 correctional 
institutions had incidences of non-compliance with security policies and procedures. 


When we followed up on the institutional action taken to rectify the identified security 
weaknesses, we found that the majority of these institutions had yet to fully implement 
corrective action. The most common security weaknesses that remained uncorrected included 
the following: 


* searches were not conducted and documented as required; 
* security control equipment and tools were not accounted for; and 
¢ fire and safety policies in response to emergency situations were not complied with. 


In our visits to the correctional institutions, we noted one institution had not completed any daily 
inspection reports as required for the month just prior to our visit. Our review showed that this 
non-compliance with the ministry requirement was reported almost a year ago by internal audit. 
At that time, the institution was completing only half of the required daily inspection reports. 


At two institutions we visited, we noted non-compliance in the suicide watch for inmates in 
segregation units. The institutions sometimes conducted suicide watches at much longer 
intervals than the required 15 minutes or less. On one occasion, we noted the interval was as 


long as 39 minutes. 
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CRITICAL INCIDENTS 


In addition to the Ministry conducting its own internal investigations when critical incidents— 
such as escapes, murders and suicides—occur in correctional institutions, they are also reported 
to police. Deaths in correctional institutions are subject to a mandatory coroner’s inquest under 
the Coroners Act. The coroner examines the circumstance surrounding the deaths and 
presents jury recommendations to the institutions where the deaths occurred. 


In the 1998/99 fiscal year, Ontario correctional institutions reported 30 escapes. In 1998/99, 
they also reported the highest number of suicides (9) in the past ten years. Our examination of 
some of these critical incident files indicated that stricter compliance with security and safety 
policies and procedures might reduce the occurrences of such critical incidents. For example: 


¢ We noted that in two of the five institutions we visited that had escapes in the 1998/99 
fiscal year, the escapes might have been prevented if security policies and procedures had 
been properly followed. In one case, a ladder was not secured as required, permitting five 
inmates to escape over the fence. In another case, inmates were not properly supervised, 
permitting four young offenders to escape during a workshop session. 


* Inone of the suicide case files we reviewed, we noted that a correctional officer did not 
observe an inmate who was under a suicide watch, as required by ministry policies and 
procedures. The inmate committed suicide when he was in the shower. In another case file 
we reviewed, the inmate committed suicide after his request to see a doctor was not 
attended to for more than two days. 





Recommendation 


To reduce or prevent critical incidents, the Ministry should more closely 
monitor the compliance with security measures by its correctional institutions 
and ensure timely, corrective action in cases of non-compliance. 





Ministry Response 


The Ministry is establishing a new performance framework that includes 
operational standards and performance measures. The Ministry is also 
reviewing and revising its current compliance monitoring tools and 
approaches to meet the new standards and to protect public safety. 


The province will have on-site monitoring teams to ensure the inmates are 
treated in an appropriate manner, while considering the interests and 
concerns of the staff at the facilities. In addition, local monitoring boards will 
| be established for all institutions, both private and public. 











INMATES WITH MENTAL DISORDERS 


Correctional institutions accommodate individuals with mental health problems on a regular 
basis. According to the Ministry, 15% to 20% of Ontario inmates require some form of clinical 
intervention for mental disorders. 
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This is consistent with our observations in the 1993 audit of Institutional Services. Following the 
de-institutionalization of treatment facilities by the Ministry of Community and Social Services 
and the Ministry of Health, the Ministry of Correctional Services noted that: 


* many of the inmates should not be in correctional institutions but should be in treatment 
facilities; and 


* correctional staff were not trained to handle inmates with mental disorders. 


The Ministry of Correctional Services Act specifies that it is the responsibility of the Ministry 
to ensure that inmates requiring hospitalization be properly placed for treatment. According to 
the Ministry, it is addressing the issues of inmates with mental disorders by: 


* Developing an information database on mentally disordered inmates as part of the 
Integrated Justice Case Management Project. This would allow for early identification of 
the mentally disordered to assist in determining possible diversion from the criminal justice 
system and/or referral to appropriate treatment/services. 


* Establishing pilot sites for implementation of the Interministerial Agreement on Court- 
Ordered Assessments. The objective is to reduce the time spent incarcerated and increase 
diversion of minor offenders from correctional institutions. 


* Working with the Ministry of Health to construct two correctional institutions on the sites of 
two existing psychiatric hospitals in Brockville and North Bay. The co-location of 
correctional and Ministry of Health treatment facilities would ensure that professional staff 
would be readily available to attend to the needs of mentally disordered inmates. At the 
time of our audit, the contracts for the two institutions had not been awarded. 


Given that the issues relating to mentally disordered inmates have been identified since 1993, 
we are concerned that the various initiatives taken to address their needs have yet to be 
implemented. 





Recommendation 














To better meet the needs of mentally disordered offenders, the Ministry 
should expedite its efforts to establish treatment facilities and diversion 
measures for such offenders. 


Ministry Response 


The Ministry is working with the ministries of Health, Attorney General, 
Solicitor General and Community and Social Services to establish treatment 
facilities and diversion measures for offenders. An inter-ministerial protocol 
has been established for clients who come into conflict with the law and are 
mentally disordered and/or developmentally disabled. 


The Ministry’s 2000/01 Business Plan includes a commitment to provide 
specialized correctional programs to reflect offenders’ sentence status, 
correctional needs and likelihood of success. This also includes the 
provision, where possible, of institutions for special treatment. The need for 
mental health and treatment services in a correctional environment will be 
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addressed by the construction of the new St. Lawrence Valley Complex which 
will provide a 100-bed secure treatment unit, together with the expansion of 
treatment beds in North Bay and Sault Ste Marie. 


MEASURING AND REPORTING ON 
EFFECTIVENESS 


As stated previously, the corporate objectives of the Ministry are to assure the protection and 
security of society and to motivate offenders towards positive personal change. We reviewed 
the indicators used to monitor, report on and evaluate the effectiveness with which the Ministry 
meets these objectives. 


The Ministry monitors and reports annually on the number of escapes of inmates under its 
custody. However, given that the majority of the offenders are sentenced for less than three 
months, protection of society by incarceration can only be a short-term measure. 


Accordingly, the ultimate protection of society rests more with how successful the Ministry is in 
achieving its other objective of motivating offenders towards positive personal change, so that 
the security of society will not be threatened when they return to the community. 


According to the Ministry, one of the most commonly used measures of positive personal 
change is the rate of recidivism (a return to crime). In our previous audits, we expressed 
concern that this effectiveness indicator was not measured. The Ministry had indicated that, 
with the implementation of its computerized offenders information system, the monitoring of 
recidivism rates of readmitted ex-offenders from the Ontario system would be significantly 
improved. 


In the 1998/99 Business Plan, the Ministry indicated it would establish a baseline by tracking the 
recidivism rate for young offenders. In our current audit’s survey of the correctional institutions, 
only one adult institution was able to provide us with recidivism statistics. At the corporate level, 
we found that the Ministry was still not measuring recidivism or monitoring the effectiveness of 
its efforts in motivating offenders towards positive personal change. 


Without reliable recidivism statistics and other effectiveness measures, the Ministry was not 
able to evaluate which programs or institutions were most effective in changing offenders’ 
behaviour. 


At the time of our audit, ministry officials indicated they were in the process of developing a 
definition of recidivism for adult and young offenders. Based on past recidivism data, a 
benchmark would subsequently be developed for measuring the effectiveness of correctional 
institutions in motivating behavioural change. 
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Recommendation 


The Ministry should develop and implement performance measures to 
assess the effectiveness of Institutional Services in motivating offenders 
towards positive personal change. 






Ministry Response 


The Ministry is establishing outcome-based performance measures to assess 
the effectiveness of programming for all sentenced offenders to determine 
the success in lowering re-offending rates. 











eS ee 


95 
Special Report: Accountability and Value for Money 





MINISTRY OF EDUCATION 


Pupil Transportation 
Grants to School Boards 





BACKGROUND 


The Education Act does not specifically require school boards to provide transportation for 
students. Rather, the Act excuses a child from attending school if transportation is not provided 
by a school board and there is no school that the child has a right to attend within the walking 
distances specified in the Act. These distances range from 1.6 kilometres for children under 
seven years of age to 4.8 kilometres for children over ten years of age. 


Although the Act does not require them to provide transportation services, all school boards in 
Ontario provide service to elementary students and most provide service to secondary students. 
The transportation of students from home to school and back is a major undertaking as 
approximately 800,000 students are eligible for service under the eligibility criteria established 
by each school board. The Ministry’s transportation grants to school boards for the year ended 
March 31, 2000 totalled $575 million, or about 4.4% of the $13.2 billion of school board funding 
entitlements. 


During the 1993 to August 31, 1999 period, total transportation expenditures across the province 
declined from approximately $622 million to $576 million or by 7%. Since the number of 
students receiving service during this time increased, the expenditure per eligible student 
experienced a slightly greater decline, from approximately $780 to $710 or 9%. 


Boards currently receive grants based on their 1997 transportation expenditures with 
adjustments for changes in enrolment. However, the primary factor influencing a school board’s 
transportation costs is not total enrolment. More important are local factors, such as enrolment 
density, which ranges from over 400 students per square kilometre in Toronto to less than one 
student per square kilometre in 25 rural, northern, and French language boards. Because of the 
differences in local circumstances, the Ministry was in the process of designing a formula for 
calculating pupil transportation grants that is based on need rather than enrolment. The Ministry 
expected to implement the new formula in time for the school boards’ 2001/02 fiscal year. 


Administration of the funding model, including grant calculations and payments to school boards, 
is the responsibility of the Ministry’s Elementary/Secondary Business and Finance Division. 
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AUDIT OBJECTIVES AND SCOPE 


Our objectives were to assess whether the Ministry had established satisfactory systems and 
procedures: 


* — to fund and deliver pupil transportation services that are economical, efficient, safe, reliable 
and in compliance with related laws, regulations and policies; and 


* to measure and report on school board and Ministry performance in this regard. 


Our audit was performed in accordance with the standards for assurance engagements, 

encompassing value for money and compliance, established by the Canadian Institute of 
Chartered Accountants, and accordingly included such tests and other procedures as we 
considered necessary in the circumstances. 


The Audit Act does not provide the Provincial Auditor with access to the information necessary 
to perform value for money audits of school boards. However, we needed to obtain an 
understanding of school board transportation policies, systems and processes in order to address 
our audit objectives. We therefore reviewed the relevant policies, systems and processes at 10 
of the 72 school boards: three English public, two French public, three English Catholic and two 
French Catholic. These boards received about 21% of the provincial transportation grants. 


We also met with the transportation managers of several other school boards. Our work at the 
boards was not an audit and consisted only of inquiry and analysis of available documentation. 


The criteria we used to perform our assessments and to reach our observations and conclusions 
were agreed to by senior management at the Ministry and the school boards that we visited. 


As the Ministry’s Internal Audit Services Branch is not permitted to examine school boards, it 
had not done any work upon which we could rely to reduce the extent of our work. 


Our work was conducted from January to June 2000. 


OVERALL AUDIT CONCLUSIONS 


The Ministry was in year three of a five-year plan to implement a revised funding and 
accountability relationship with school boards. It had not yet established the systems and 
procedures needed to ensure the economical and efficient delivery of pupil transportation 
services. Although we found examples of good practices at most of the school boards we 
visited, the Ministry had not yet established mechanisms to encourage the adoption of such 
practices by all boards. Most transportation managers we met with acknowledged that further 
efficiencies could be achieved by adopting proven changes made at other boards. However, 
some stated that their boards were reluctant to implement further efficiencies until the Ministry 
finalized the formula for calculating transportation grants. Boards wanted assurance that any 
savings they realized would accrue to them and not to the Ministry. 


Making informed decisions about transportation policies requires reliable information about the 


impact of different service levels on costs. However, we found a serious lack of meaningful 
information on the operations and performance of school board transportation Services, as 
neither the Ministry, nor the trustees at most school boards that we visited, required such 
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information to be collected and reported. In particular, there was insufficient information to 
enable either trustees or the Ministry to relate costs to the level of service provided, which 
varied significantly among the boards we visited. As a result: 


* neither the Ministry nor board trustees were in a position to compare the current year’s 
performance of board transportation services to past years or to those of other boards, or to 
assess and report on the economy and efficiency with which transportation services were 
delivered; and 


* because the Ministry still lacks the information required to develop a needs-based formula 
for determining pupil transportation grants, finalization of the formula has been delayed. 


We also concluded that: 


* — significant inconsistencies in transportation services and policies existed among school 
boards, which warrants a review and clarification of the legal obligations and policy 
requirements that govern the provision of pupil transportation services; 


¢ — little information was available to enable boards to track and manage the significant 
proportion of transportation costs incurred to provide transportation services to high-needs 
students; and 


¢ funding inequities will persist if transportation grants are used to avoid safety hazards that 
are normally a municipal responsibility to address. 


DETAILED AUDIT OBSERVATIONS 


EFFORTS TO REDUCE TRANSPORTATION 
COSTS 


We met with three transportation managers whose boards had taken a number of initiatives to 
reduce costs and who, because their boards not been subject to amalgamations, were able to 
provide us with historical information about their cost reduction efforts, which are summarized 
in the table below. 


Reduction in Pupil Transportation Costs 


Board/Consortium Provincial 
pvsiets 













Expenditures in 1993 ($ 000) 19,400 38,900 5,980 622,000 
Expenditures in 1999 ($ 000) 16,800 29,600 4,590 576,000 
Percentage Change (13%) (24%) (23%) (7%) 


Per Eligible Pupil 1993 
Per Eligible Pupil 1999 560 720 
Percentage Change (27%) (14%) (29%) (9%) 























Sources: Three school boards not affected by amalgamation 
and the Ministry of Education 
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The savings initiatives implemented by these boards included: 


* reducing the number of buses required by staggering school operating hours, increasing 
cooperation with neighbouring boards, planning services based on actual rather than eligible 
ridership and using route planning software to optimize bus route networks; 


* — reducing service, such as eliminating home pick-ups; 
* altering kindergarten programs to eliminate midday busing; and 
* reducing rates paid to school bus operators. 


As the table shows, the percentage reduction in both total and per-eligible-pupil transportation 
expenditures achieved by these boards was significantly higher than the provincial average. 
Transportation managers we met with at other boards acknowledged that further efficiencies 
could be achieved by adopting proven changes made elsewhere and by experimenting with new 
strategies, such as integrating school bus service with municipal transit services. However, 
some transportation managers indicated that their boards were reluctant to implement further 
efficiencies until the Ministry finalized the design of the transportation grants in order to ensure 
that any savings they realized would accrue to them and not to the Ministry. In their view, 
because the Ministry is funding boards at their 1997 level of transportation expenditures, boards 
that made significant efforts to reduce their costs prior to 1997 were being treated inequitably 
relative to boards that had not made similar improvements. 


The efficiency and economy of pupil transportation services was considered by the Ministry’s 
Transportation Funding Review Committee and by the Education Improvement Commission. 
The latter body was established by the Minister in 1997 to help ensure that Ontario’s new 
district school boards were established in an organized and appropriate manner. 


One of the key recommendations of the Transportation Funding Committee was that a new 
needs-based funding formula be developed. The Ministry initially committed to having the new 
formula done by February 2000, but, due to changing policy priorities, this has now been 
targeted for completion by February 2001. The delay has also resulted from the lack of 
information needed to complete this critical project. 


Both the Transportation Funding Committee and the Education Improvement Commission also 
recommended that all boards deliver transportation services through local “consortia,” whereby 
boards serving the same area jointly provide transportation services. Based on our visits and 
discussions, which included boards that participate in a consortium, we support that 
recommendation. In addition to the potential for savings from greater sharing and integration of 
routes, we were told that consortia offer a number of other benefits including: 


* amore consistent level of service to all area parents and students regardless of the board 
they support or who their trustee happens to be; 


* economies of scale that result in more cost-effective administration; and 


* — sufficient scale of operations to establish effective performance monitoring and reporting 
functions for both school bus operators and transportation services overall. 
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Recommendation 


To help ensure that school boards implement best practices to deliver 
transportation services as cost-effectively as possible, the Ministry should: 





e expedite the development of the new funding formula; 


¢ ensure that the new funding formula contains incentives for boards to 
implement cost saving measures; and 


¢ establish target dates for school boards that have not already done so to 
either form transportation consortia or submit a business case 
demonstrating why it would not be beneficial for them to integrate the 
delivery of their transportation services. 


Ministry Response 


The Ministry is committed to developing a new funding approach for the 
2001/02 school year. 


Incentives for cost reduction will be assessed as options in the Ministry’s 
funding approach. For example, the Ministry is considering options for using 
the funding approach to encourage boards to form consortia. Funding 
incentives/disincentives will be more effective than establishing target dates 
to form consortia. 








INFORMATION FOR DECISION MAKING 
INFORMATION ON POLICY ALTERNATIVES 


In addition to eligibility criteria, there are a number of policy decisions that school boards make 
that have a significant impact on the cost of providing transportation services. For example, 
boards: 


¢ determine the extent to which bell times among their schools should be staggered to enable 
buses to be used for more than one school (bell times are the times at which classes start in 
the morning and end in the afternoon); 


* — set window times (windows are the maximum interval between morning drop-off and the 
starting bell time, and afternoon pick-up and the ending bell time); and 


* — set the maximum run time (the run time is the length of time that the first student to be 
picked up spends on the bus). 


The following table summarizes the decisions made by the boards we visited with respect to 
these policies. 


a 
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Sample of Transportation Policies at School Boards Visited 


Board/Consortium 


Bell Time Range (minutes) 
¢ Elementary 
¢ Secondary 








Maximum Run Time (minutes) 
e Elementary 
¢ Secondary 





Source: School boards visited 


In each case trustees must balance costs against the reasonableness of alternative policies for 
students, parents and staff. Longer ranges for staggering bell times and longer window times 
help boards reduce the number of buses needed by enabling buses to be used on more than one 
run—for example, delivering students to one school at 8:00 a.m. and a second school at 
8:30 a.m. Longer run times enable boards to reduce the number of buses by enabling each 
bus to pick up more students. The price of these initiatives, however, is less convenience for 
parents and students. 


Making informed policy decisions on these matters requires reliable information about the 
impact of each alternative on costs, but only one board visited was able to produce such 
information. In connection with their review of bell times and windows, trustees at this board 
wanted cost information about four alternatives. Management was able to provide trustees with 
the impact of each alternative on transportation expenditures, which ranged from a saving of 
approximately $200,000 to additional costs of $320,000 per year. 


Another board that was not in our sample provided us with information about cost saving 
initiatives that it had provided to its constituents and trustees. Transportation management was 
able to provide trustees with the savings attributable to each of several initiatives taken by the 
board to reduce costs, such as route optimization and staggering bell times. These savings 
amounted to over 10% of the board’s transportation budget. 


Most of the boards we visited had difficulty providing trustees with such information due to the 
extent of manual work involved in developing their bus route networks. To identify the impact 
of different alternatives on costs, the entire route network must be reworked for each 
alternative. This is a time consuming and expensive task unless a board relies almost completely 
on transportation software to design its network. However, only one of the boards we visited 
was in this position. The others advised us that transportation planning personnel determined 
20% to 30% of the actual network of routes used. Consequently, the information provided to 
trustees to enable them to make informed transportation policy decisions was limited at most 
boards visited. 


Informed decision making and public debate is also assisted by publishing the rationale 
underlying each policy. However, none of the boards we visited had performed or documented 
any assessment of the reasonableness of the policies in relation to the affected students. 
Instead, the transportation managers at these boards advised us that the level of service was 
primarily determined by the amount of the transportation grant and, to a lesser extent, the 
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practices at nearby boards. Thus, boards that spent comparatively more on transportation prior 
to 1998, and, as a result, received comparatively generous funding from the Ministry, could 
afford a higher level of service than other boards. 


PERFORMANCE INFORMATION 


The Ministry had not obtained and analyzed transportation operating data from school boards. 
Consequently, the Ministry was not in a position to: 


* — establish benchmarks against which trustees can assess the efficiency/economy of 
transportation services; and 


* identify and promote best transportation practices. 


The information required by trustees in order to meet their responsibility to monitor the 
performance of their board’s transportation operations is also required by the Ministry to 
develop a needs-based formula for calculating transportation grants. Some examples of 
information that most boards we visited did not provide to trustees included: 


* Ridership percentage: Most boards did not regularly count riders and track the percentage 
of eligible students who used the service. In these boards the number of buses needed was 
determined based on eligible rather than actual riders. Boards that have performed counts 
discovered that the ridership percentage was often much less than 100% and have been 
able to reduce costs by more closely aligning capacity with actual use. For example, one 
board stated that in doing so, it was able to reduce its costs by approximately 10%. In 
addition to identifying opportunities to reduce costs, monitoring trends in the ridership 
percentage may provide trustees with evidence of the degree of satisfaction with the 
service. 


* Reliable capacity utilization measures: A number of factors influence a board’s overall 
capacity to safely transport students, including the size and number of buses used, the age 
of students and the potential for multiple runs with the same bus. Trustees at the boards we 
visited had not received meaningful and reliable information about actual capacity and 
ridership. Consequently, they were not in a position to compare their board’s utilization of 
buses to other boards or assess how efficiently their board was using buses. 


* The costs and benefits of performing transportation planning and administration in-house 
versus outsourcing these functions: Although all the boards we visited had reduced costs by 
using third parties to deliver bus services, only two had investigated the use of third parties 
to deliver planning and administration services and both had decided to outsource these 
services. 


* A breakdown of service deficiencies and complaints by category and an analysis of year- 
over-year trends: Two of the boards that we visited advised us that they would start 
providing this performance measure to trustees in connection with their implementation of 
updated versions of their transportation software. 
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¢ The results of performance audits of their service providers: Most boards we visited either 
did not conduct performance audits of bus or taxi operators, or did so ina sporadic, informal 
manner without standard audit programs or reporting of results. Such audits would include, 
for example, determining the level of compliance with safety provisions in contracts, such 
as the adequacy of driver training programs, and verifying the appropriateness of kilometre 
charges. Trustees did not receive information about such matters and consequently, were 
not in a position to monitor year-over-year trends. 


* Information that relates costs to the level of service provided: The annual expenditure 
information that trustees were receiving was not sufficient to enable them to assess the 
efficiency and economy of their board’s transportation services. For example, 
transportation management can reduce year-over-year expenditures by reducing service, 
such as eliminating service for secondary students. However, such action does not increase 
efficiency: it is simply less service for less money. On the other hand, reducing the number 
of buses needed while serving the same number of students, perhaps by optimizing the 
route network, does represent an increase in efficiency. 


Without information that relates costs to level of service, trustees cannot compare the 
current year’s operations to those of past years. Such information would also enable the 
Ministry to establish benchmarks against which it, trustees, and the public can assess the 
relative efficiency and economy of board transportation services. It would also enable the 
Ministry to identify and promote best transportation practices. 


Se 


















Recommendation 


To facilitate prudent decision-making and strengthen local accountability, the 

Ministry should: | 

¢ ensure that boards acquire and utilize information systems that can 
provide the information needed to enable trustees to make informed 

_ decisions about the level of transportation services offered; and 

* require boards to develop reports that relate costs to level of service in 
order to ensure that the Ministry and trustees can compare boards to 
each other and to established benchmarks. 


Ministry Response 


The Ministry’s new funding approach will require boards to implement 
student transportation management software. As a result, boards will have 
more useful and accurate information. 


In the meantime, the Ministry is developing a survey to collect the data 
needed for decision making. The Ministry also announced, in March 2000, an 
interest-free loan program to assist boards to implement student 
transportation management software. 
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As part of its new funding approach, the Ministry is developing cost 
benchmarks. Also, the required student transportation management software 
will give boards a greater capacity to evaluate the cost impacts of different 
service levels. 


In July 2000, the Ministry released a Request for Qualification for student 
transportation management software to ensure that the products that boards 
acquire will meet the Ministry’s information needs and that the information 
obtained is reliable for use in determining boards’ needs. 








CLARIFYING LEGAL AND POLICY 
REQUIREMENTS 


Transportation grants to school boards have been unconditional since 1995. For example, school 
boards are not required to implement certain minimum eligibility and service policies that are 
designed to provide a basic or core level of service. In fact, there is no ministry requirement to 
provide any service at all. The Ministry advised us that whether a board spends all, some or 
none of the grant on transportation is a matter for trustees and the local community to decide 
and that this will continue to be the case under the new formula for calculating transportation 
grants. 


Although the new formula will not include a requirement to provide a core level of service, it is 
the Ministry’s objective to provide each board with an equitable share of the available 
transportation funding such that each board is in the position to provide a core service. It 
intends to do this by computing a “needs index”’ for each board using commercially available 
transportation software created specifically for school boards. These software packages use 
digitized area maps, student data, such as their addresses and the schools they attend, and 
various policy inputs, including walking distances, maximum run times and bell times, to design 
the bus route network that minimizes the number of buses that a board requires. This number 
will be a key factor in determining a “needs index” for each board. Grants will be calculated by 
multiplying the indexes by a ministry-determined standard cost factor. 


Although the Ministry expects trustees and the local communities to debate what portion of the 
grants should be spent on transportation services, we did not find evidence that such debate 
was taking place. Instead, as the table below shows, the level of service provided was related 
to each board’s grant. For example, some boards eliminated service to older students in order to 
stay within their grant level. 
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Pupils per km? 


Transportation Service Eligibility and Expenditures 
of School Boards Visited 
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Percentage of Students Eligible for Service 








Expenditures Per Enrolled Student 





Expenditures Per Student Eligible for Service 
Walking Distances (km): 
i ANIL te BEL: : 
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Grades 7 and 8: Urban hs i | Offered 


PeeeGareendemuals 5 | 468 | 46 | so.) if. 
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Grades 2-OAG: Urban | Offered | offered | otferea | 32 | 
Grades 9—OAC: R ural ; i 40 





Source: School boards visited 


This is consistent with what transportation managers told us about their mandate: their trustees 
expected them to provide as much service as possible within the grant amount. We saw no 
evidence at these boards that the transportation grant was viewed as being available for 
reallocation to other programs if it could be demonstrated that such programs were of higher 
priority than transportation services. None of them published information that was specifically 
designed to assist trustees and the public in assessing the relative priority of their board’s 
various programs. The Ministry has not provided boards with any guidance about the type of 
information they should publish in order to facilitate public debate about spending priorities. 


We also found a lack of guidance from the Ministry to boards about their legal, legislative and 
regulatory obligations with respect to student transportation services. We noted that: 


While most boards we visited understood that, under the Education Act, transportation 1s a 
privilege and not a right, some boards believed that parents could successfully sue their 
board if it did not provide service to students living outside the walking distances specified 
in the Act. Trustees and local communities cannot properly debate the level of service to 
provide if there is uncertainty about the boards’ legal obligations. 


The Education Act does not permit boards to recover costs from those using the service, 
as is allowed in Alberta for example, because there is no specific provision in the Act that 
allows school boards to charge fees for such services. As a result, we found that: 


Urban secondary students were treated inequitably vis-a-vis rural and suburban 
students within the same board. At some boards, urban secondary students must pay to 
use municipal transit systems whereas the board provides free service to rural and 
suburban students. Although the Ministry is not opposed to the elimination of service 
for urban secondary students, it would oppose a board raising, through service charges, 
the funds required to provide equitable service to all secondary students. 
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¢ There was little flexibility in meeting the needs of parents who were willing to 
reimburse the school board for the cost of additional service, such as transportation to 
after-school day care for elementary students. However, recovering the costs of such 
non-core service would conflict with the Ministry’s position that services should be 
provided free or not at all. 





Recommendation 






To help ensure that school boards provide transportation services that meet 
local needs in an equitable manner and adhere to policy and legal 
requirements, the Ministry should: 












¢ provide guidance to boards on the spending and service level 
information that they should publish in order to facilitate informed public 
debate among trustees and their community; and 


¢ clarify the legal and policy parameters that it expects boards to operate 
within. 


Ministry Response 


The Ministry’s long-term goal is to increase public reporting in all aspects of 

education, including transportation. The student transportation management 
software will provide boards with the information necessary to meet ministry 
reporting requirements and to facilitate informed public debate. 


Part of the Ministry’s overall strategy is to base funding levels on uniform 

policy parameters (such as walking distances). These policy parameters will 
serve as benchmarks and help boards make appropriate decisions about 
| service levels. 











TRANSPORTATION OF SPECIAL NEEDS 
STUDENTS 


All of the boards that we visited believed that the cost of transporting students with special 
needs was a significant portion of their overall transportation expenditures. Among those boards 
that had the necessary cost information, it ranged from 18% to 40% of transportation 
expenditures, as shown in the following table. 
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Special Education Transportation Data for School Boards Visited 


Percentage of Enrolment Designated as 

Percentage of Transportation Expenditures on 

Special Education wate oo 35.2 
Annual Per Pupil Cost — Special Education $3,380 | $4,460 | $2,310 | $3,520 | 


Note: Only four boards visited could provide us with special education transportation information and 
we have not verified its accuracy. 


















The cost for each special needs student ranged from a few hundred dollars per year, for those 
that could be transported with general education students, to several thousand dollars per year 
for high-needs students, where taxis or specialized vehicles had to be used. For a few very 
high-needs students, the annual cost per student was several tens of thousands of dollars. 


As the table above illustrates, the incidence of special needs students and the cost of 
transporting them varies significantly among boards. In order to design an equitable formula for 
calculating transportation grants, the Ministry must take into account differences in special 
needs transportation requirements. One way for the Ministry to accomplish this would be to 
move a portion of the Transportation Grant into the Special Education Grant. 


Such an approach would also encourage boards to correct another problem we encountered. 
Superintendents responsible for special education have de facto spending authority for the 
transportation of special needs students without being accountable for the costs: they arrange 
for student transportation through the board’s transportation manager and the costs are 
absorbed by the regular transportation budget. We noted that at the boards we visited only one 
superintendent who was responsible for special education regularly received cost information. 


As a result of this split in spending authority and accountability for costs, neither the trustees 
nor the Ministry have assurance that all costs, including transportation, are considered in 
designing and locating special education programs or in making program choices. Correcting the 
split would help ensure that the most economical method is selected to achieve the educational 
objectives for each special needs child. 











Recommendation 
To enhance accountability and to ensure equitable funding of transportation 
for special needs students, the Ministry should: 


¢ require boards to track and report these costs separately; and 
* consider funding transportation costs for high-needs students through 
the Special Education Grant. 
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Ministry Response 


With the implementation of student transportation management software, 
reliable operating and cost data will be more readily available. In the 
meantime, the Ministry’s survey to collect the data needed for 


decision-making will include gathering information about the transportation 
of special needs students. 


The Ministry will consider the option of including funding for special 
education transportation under the Special Education Grant as part of the 
new approach to funding student transportation. 





FUNDING POLICY FOR SAFETY HAZARDS 


Many students, who would not otherwise be eligible, receive transportation services due to the 
existence of safety hazards, the criteria for which vary from board to board. Examples of 
hazards include busy intersections, lack of sidewalks, and physical barriers like ravines. 


Neither the Ministry nor the boards have information about the impact of hazards on 
transportation costs. Only three boards visited had tracked the proportion of students they 
transported because of safety concerns. At those boards, the proportion ranged from 14% to 
36% of the total number of students transported. Such a wide variation can have a significant 
impact on transportation costs and, therefore, on ministry funding requirements. 


Most safety hazards of which we were advised are the responsibility of municipal governments 
to manage and can be corrected, for example, by employing crossing guards or installing traffic 
lights, sidewalks or pedestrian bridges. The practices of boards in dealing with hazards vary 
across the province. Some provide transportation services to all students affected by hazards; 
some advise parents that it is up to the municipality to eliminate hazards and refuse to use 
education grants to cover municipal responsibilities; and others take a case by case approach. 


However, none of the boards we visited had worked with municipalities to develop a 
mechanism whereby the costs and benefits of removing a hazard versus incurring the cost of 
busing were analyzed and the more cost-effective option taken. A more consistent level of 
effort by boards and municipalities to find cost-effective solutions to address safety hazards 
could result in substantial savings. According to the boards we visited, each bus that can be 
eliminated by correcting hazards would save them $30,000 annually. 


The Ministry advised us that it intends to factor hazards into the new formula for calculating 
transportation grants, although it had not yet established the criteria for determining what 
constitutes a fundable hazard. Our concerns are as follows: 


* Incorporating hazards into the funding mechanism would be inequitable for jurisdictions 
where municipalities have eliminated hazards at the request or insistence of local school 
boards. 


¢ Such funding would eliminate any incentive for boards to require municipalities to eliminate 
hazards as the savings would accrue to the Ministry. Thus, taxpayers would lose in 
situations where eliminating a hazard is more cost-effective than busing. 
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* Arriving at a consistent province-wide definition of a safety hazard for funding purposes 
may be difficult and lead to disagreements over whether the Ministry is providing sufficient 
funds for boards to ensure the safety of students. 








Recommendation 














To better ensure that transportation grants are equitable and utilized in a 
cost-effective manner, the Ministry should: 


* clearly specify the conditions, if any, under which it is appropriate for 
school boards to use transportation grants to address municipal 
responsibilities; and 

* ensure that the funding mechanism provides incentives for school 
boards and municipalities to work together to implement the most 
economical solutions for addressing safety hazards. 


Ministry Response 


The Ministry agrees that responsibility for safety hazards is a significant issue 
that must be addressed in the funding review. There is a need to achieve 
greater consistency among boards in the definition of a safety hazard for 
transportation purposes. 


The review process is exploring options to ensure that there is an appropriate 
distribution of responsibility and accountability for the safety of pupils while 
en route to and from school. 
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MINISTRY OF THE ENVIRONMENT 


Operations Division 





BACKGROUND 


Under the Environmental Protection Act, Ontario Water Resources Act, Environmental 
Assessment Act, Pesticides Act and other related acts and regulations, the Ministry of the 
Environment is charged with a broad mandate of protecting the quality of the natural 
environment in order to safeguard the ecosystem and human health. 


The Operations Division carries out its responsibilities by administering the Ministry’s approvals 
and enforcement activities. It also responds to reports of pollution and spills that may have 
health and environmental impacts. In addition, the Division may clean up abandoned 
contaminated sites using funds available in the Environmental Clean-Up Fund. 


The Division operates through a province-wide network of inspection, technical and 
investigative staff in regional, district and area offices; a 24-hour Spills Action Centre; a Smog 
Patrol; and a centralized approvals and environmental assessment office. 


For the 1999/2000 fiscal year, the Division had total expenditures of $62 million and about 680 
staff. Since 1994, the Division has reduced its staff level by over 25%. 


AUDIT OBJECTIVES AND SCOPE 


Our audit objectives were to assess whether the Ministry had satisfactory systems and 
procedures in place to: 


* administer approvals and enforce compliance with environmental legislation; 


* measure and report on the performance of the Division in contributing to the effectiveness 
of the Ministry in protecting the environment and human health; and 


* — ensure that the Division’s resources were managed with due regard for economy and 
efficiency. 


The scope of our audit, which was substantially completed in March 2000, included interviews 
with appropriate staff and review and analysis of policies and procedures, management reports, 
samples of files and financial and management systems. We also researched practices in other 
jurisdictions. We did not rely on the Ministry’s internal auditors to reduce the extent of our audit 
work because they had not recently conducted work within the scope of our audit. 


Our audit did not include the Environmental Assessment or Conservation and Prevention 
programs which, as a result of a recent ministry reorganization, are now responsibilities of the 
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Operations Division. These areas were covered as part of our 1997 audit of the former 
Conservation and Prevention Division. 


Prior to commencing the audit, we identified the audit criteria that would be used to conclude on 
our audit objectives. These were reviewed and accepted by senior ministry management. 


Our audit was conducted in accordance with the standards for assurance engagements, 
encompassing value for money and compliance, established by the Canadian Institute of 
Chartered Accountants, and accordingly included such tests and other procedures as we 
considered necessary in the circumstances. 


OVERALL AUDIT CONCLUSIONS 


We concluded that the Ministry did not have satisfactory systems and procedures in place to 
administer approvals and enforce compliance with environmental legislation. A number of our 
observations resulted from the Ministry not having appropriate systems and/or information to 
properly support the Division’s inspection and enforcement activities. Our major concerns 
included: 


¢ The Ministry’s systems were inadequate for assessing whether and to what extent the over 
220,000 certificates of approval issued since 1957 needed to be updated with new 
conditions and requirements. (A certificate of approval is required for any facility that 
discharges contaminants into the environment.) As a result, it did not know the extent to 
which facilities were not meeting current environmental standards. 


¢ Over $90 million in financial assurance, such as cash or collateral, was not obtained from 
facility operators as required under legislation and ministry policy. This poses significant 
financial risk to the province should the operators become insolvent and government funds 
then be required to clean up damages to the environment caused by the operators. 


¢ A reduction in staff of 25% over the last four years had contributed to a 34% decrease in 
the number of ministry-initiated inspections conducted per year. Further, the Ministry 
identified significant violations in 31% of the inspections conducted. The rate of non- 
compliance would have been even higher had many violations the Ministry considered 
minor been more appropriately treated as significant. 


* The Ministry relied extensively on facility operators to comply voluntarily rather than 
impose stringent enforcement measures, such as issuing control orders or laying charges. 
This was of particular concern as one third of violators were repeat offenders. In addition, 
the Ministry did not appropriately follow up on many violations to ensure that deficiencies 
had been corrected. 


¢ Environmental fines imposed on violators have averaged $1.5 million per year. Over 
$10 million in fines that had accumulated over many years remained unpaid. In order to 
more effectively enforce payment and environmental legislation, the Ministry needs to more 
aggressively advise operators that it will use its statutory authority to suspend their 
operations. 
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¢ The Ministry typically learned of contaminated sites only after serious harm to the 
environment had already occurred. This made it difficult to hold facility operators 
responsible for the damages where significant clean-up costs were required and long time 
periods had elapsed. The Ministry needs to develop a strategy for early identification of 
high-risk contaminated sites to allow for better planning and prioritization of clean-up 
efforts. 


The Ministry, having recognized many of these problems, had conducted a number of internal 
reviews—the results of which were consistent with the findings in our audit—and was in the 
process of addressing these issues. 


To demonstrate its progress in managing the environment, we concluded that the Ministry 
needed to measure and report on its performance in a comprehensive and objective manner. 





Overall Ministry Response 


The report on the audit of the Operations Division offers many constructive 
comments and recommendations regarding the Division’s role in delivering 
the Ministry’s mandate for protection and conservation of the natural 
environment. The Operations Division, over the past two years, has initiated 
an aggressive in-depth review of its practices and procedures and is actively 
implementing action plans based on this work. The Ministry is pleased to 
note that many of the recommendations of the report complement a number 
of the initiatives the Ministry is pursuing. Work is well underway on several 
key components of these recommendations, while other items have phased 
review and implementation plans that involve the whole Ministry. 











DETAILED AUDIT OBSERVATIONS 


COMPLIANCE WITH ENVIRONMENTAL 
LEGISLATION 


CERTIFICATES OF APPROVAL 


Under the Environmental Protection Act and the Ontario Water Resources Act, a certificate 
of approval is required from the Ministry for any facility that discharges contaminants into the 
environment. Certificates of approval are issued for waste, water and sewage treatment 
facilities, and for facilities that may emit a contaminant into the air. Facilities may have multiple 
certificates of approval for various systems and equipment used in their operations. The 
Ministry is responsible for reviewing and approving almost 8,000 applications each year. 


Certificates of approval document site-specific requirements, including a description of the 
undertaking, the engineering principles of the site, system or process, and the controls and 
contingencies. For requirements that are not already specified in an act or regulation, the 
certificates of approval are used as a compliance mechanism to relate standards and pollution 
parameters to applicants in a legally binding manner. Legislation provides significant discretion 





112 Office of the Provincial Auditor 











eeeeeseseeSFSsSsSseses 


to the Ministry to impose operating, financial and reporting requirements as a condition of 
approval. 


UPDATING CERTIFICATES OF APPROVAL 


Since 1957, when certificates of approvals were first introduced, the Ministry has issued over 
220,000 approvals. Certificates of approval generally do not have expiry dates or renewal 
requirements. They remain in effect until a request is made by the facility operator to change its 
process, the facility closes, or the Ministry initiates changes or, because of violation problems, 
decides to revoke or suspend the certificate of approval. 


We found that the Ministry did not have an adequate system in place to review the terms and 
conditions of the existing certificates of approval to ensure they met current environmental 
standards: 


¢ Approvals issued before 1986, which totalled about 130,000, were recorded on a manual 
card index system. The manual system made it impractical to summarize or analyze the 
conditions of the approvals and to determine whether these approvals had expired or been 
revoked or updated. 


¢ Approximately 90,000 approvals issued between 1986 and October 1999 were stored within 
various databases. For many of these approvals, the conditions were not described in detail 
or were not recorded in the databases. We also identified instances where the certificate of 
approval should not have been listed as active, such as when a new certificate had replaced 
an earlier one. 


Over time, there have been many amendments to legislation and ministry policies and 
guidelines. These have resulted in more stringent conditions attached to certificates of approval 
that require greater accountability and due care by the owner or operator of a facility. For 
example, new certificates of approval may limit operational capacity, effluent and discharge 
parameters, and require either periodic reporting of water and air quality test results to the 
Ministry or making the results available upon inspection. As changes were made, existing 
approvals were generally not adjusted although legislation provides the Ministry with the 
authority to do so. For instance, approvals granted before 1983 included few, if any, conditions. 


The Ministry’s management advised us that updating existing approvals would involve a 
significant workload and expense for facility operators and the Ministry. Generally, the 
Ministry’s approach has been to apply changes only to new certificates of approval or to 
certificates being amended when facilities or sites initiate any adjustments to their process or 
structure. 


As a result, many owners and facilities were operating with certificates of approvals that did 
not meet current standards for such approvals and inconsistencies existed between site 
operators operating under the same environmental conditions. A system for updating approvals 
where deemed necessary would help ensure that legislative and ministry requirements are being 
applied consistently and that existing systems are operating as originally intended. 


Ministry management had initiated an internal assessment of its approval process to evaluate 


different approaches for ensuring that certificates of approval are updated to reflect current 
conditions. 
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Recommendation 


In order to ensure that existing certificates of approvals reflect current 
environmental standards, where required, and are being applied 
consistently, the Ministry should: 


* improve its information systems so that all certificates of approval can be 
assessed on the extent to which they need to be updated with new 
conditions and requirements; 


* develop systems that would allow for updating certificates of approval in 
a timely and efficient manner; and 


* establish action plans and timetables for when certificates of approval will 
be required to be up-to-date. 





Ministry Response 


We agree. The Ministry is fundamentally changing the way certificates of 
approval are issued and amended. The Program Effectiveness Review 
currently in progress will develop options to ensure certificates of approval 
are current and updated. The terms of reference have been modified to 
incorporate the Provincial Auditor’s recommendations. 


Reviews of priority sectors, such as water and hazardous waste, have already 
been undertaken or are in progress to improve certificates’ currency, 
accuracy and compliance. For example, approvals for municipal water 
treatment plants will now be consolidated into a single-site document that 
will be reviewed and renewed within a three-year cycle. 


The Integrated Divisional System (IDS), once fully implemented, will enable 
the Ministry to assess over time the extent to which certificates of approval 
need to be updated. The matter of populating this system with information on 
historical certificates of approval, the resource impacts, and alternate options 
to fulfill the need identified will be addressed in the Program Effectiveness 
Review. 














FINANCIAL ASSURANCE 


Under legislation passed in 1986, the Ministry may require as a condition of approval that 
facility operators provide financial assurance, such as cash or letter of credit, to the Ministry as 
security. The purpose of financial assurance is to indemnify the Ministry against losses such 
that funds are available, if needed, for rehabilitating or cleaning up sites should the operator be 
unable or unwilling to do so. The Ministry has established policies that identify higher-risk 
activities for which financial assurance is a mandatory requirement (for example, private landfill 
sites) and those for which it can be required at the discretion of the program director. 


The policies also outline procedures, primarily based on risk, for calculating the amount of 
security required. Depending on the type of operation, the security required could be used to 
cover the costs of long-term monitoring of the site, clean-up of the property or spills, temporary 
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operating costs, and for providing alternative water supplies. The amount of financial assurance 
averages about $120,000 and can be as high as several million dollars. Financial assurance is 
returned to the applicant once the terms of the approval or order are met or upon the 
termination of operations. As of March 31, 2000, the Ministry held securities totalling 
approximately $98 million. 


We concluded that the Division’s controls over obtaining financial assurance for approvals were 
inadequate and posed significant financial risk to the province should companies fail to meet 
their obligations or become insolvent. Specifically: 


¢ During 1999, the Ministry conducted a division-wide assessment of the financial assurance 
obtained for certain types of approvals. It found that, of about 1,100 approvals, financial 
assurance was not obtained as required in about 710 or 65% of the cases. In about half of 
these 710 approvals, the requirement for financial assurance was not specified in the 
certificates of approval. While the Ministry’s information systems did not allow for a more 
precise estimate of how much financial assurance was not obtained as required, we 
estimated the amount to be over $90 million. 


¢ We noted several cases where the amount of financial assurance required was not 
sufficient to cover clean-up costs in the event of non-performance by the facility operator. 
For example, in one recent case where an operator experienced financial difficulties, the 
potential clean-up costs were estimated at $1 million to $2 million in April 2000 whereas the 
financial assurance required and obtained was only $38,200. 


¢ Ministry policy for the types of securities accepted was also not followed. The policy states 
that in certain situations only cash or government bonds can be accepted as financial 
assurance. However, letters of credit and surety bonds were regularly accepted for such 
approvals even though these types of securities can be cancelled by the operator with only 
two months’ notice. 




















Recommendation 

To minimize financial risk to the province relating to environmental clean-up 

costs, the Ministry should: 

¢ establish controls to ensure financial assurance requirements are 
assessed and specified in the certificates of approval and then followed; 
and 

* identify, for each facility operator, the correct amount of financial 

assurance outstanding and take timely action to obtain the necessary 

assurance. 


Ministry Response 


The Ministry’s internal review of its financial assurance requirements has also 
demonstrated deficiencies in the collection of financial assurance _ 
requirements, and corrective action has been implemented in financial 
assurance administrative procedures for ensuring financial assurance 


requirements are met in the future. In addition, the Ministry is undertaking a | 
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review of its financial assurance policy, including appropriate forms of 
security. 






An action plan to rectify outstanding financial assurance requirements has 
been put in place. 








COSTS OF MONITORING COMPLIANCE 


The large volumes of over 220,000 existing certificates of approval and almost 8,000 new 
certificates issued each year make it impractical for ministry staff to closely monitor all site 
operators for compliance with the conditions of their approvals. Where monitoring does take 
place, the costs to monitor site operators for compliance can be significant, particularly for large 
operations. For example, we estimated it cost the Ministry $80,000 per year in environmental 
officer salaries to monitor just one mining operation. Therefore, there is a need to ensure that 
conditions of approval include self-monitoring requirements for approval holders to report on 
their performance and demonstrate their compliance to the Ministry. Though the Ministry has, 
in some cases, already imposed such requirements, we believe the practice could be expanded: 


* Comprehensive monitoring and reporting requirements have already been established or are 
planned for large companies that discharge contaminants into the waterways and air. For 
example, under the Municipal/Industrial Strategy for Abatement (MISA) program, 
regulatory requirements were established in 1993 and 1995 for approximately 190 large, 
municipal, industrial mining and electricity producing companies that discharge wastewater 
into Ontario’s waterways. These operators are required to conduct and pay for regular 
testing of their effluents to ensure compliance with discharge parameters and to make their 
testing public upon request. 


However, these comprehensive regulations generally apply to the largest operators only, 
which represent a small number of the total certificates of approval granted. 


* Fora few large landfill sites and demonstration pilot projects, the Ministry required the 
applicants to pay for the monitoring costs of either full-time or part-time independent 
inspectors since monitoring these operations would have placed large demands on ministry 
resources. However, the vast majority of approvals for landfill sites and industrial 
operations did not include this requirement. 


We noted that on October 27, 1999, a decision by the Environmental Appeal Board 
recommended that the Ministry consider requiring all landfill site operators to pay for the 
cost of independent inspectors. 


* Once a certificate of approval has been issued for a system or equipment, there are no 
further requirements for the approval holder to periodically submit an audit or assessment 
by an independent expert that the system is operating as intended. Furthermore, we noted 
that Ministry staff generally did not conduct site visits during the application period and 
relied on reports from the owner that the installation was made in accordance with the 
approval requirements. 
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Requiring independent experts to certify systems and equipment both at the time of 
installation and at periodic intervals thereafter would help ensure that conditions of approval 


and ministry standards are met and would significantly reduce the Ministry’s inspection 
requirements for these facilities. 


* For facilities that discharge significant air pollution, the Ministry has established and 
operates air monitoring stations in their vicinity to monitor compliance. There are 137 
stations in 24 communities monitoring about 50 facilities. Furthermore, eight of these 
facilities receive data from the stations. In a few cases the Ministry has been successful in 
establishing partnerships with facility operators to help pay for the air monitoring station 
costs. For example, in one area the Ministry pays for only 20% of the approximately 
$200,000 annual operating costs for air monitoring stations. 


We were advised that the Ministry is developing a strategy, to be completed by January 
2001, to have facility operators pay for all operating and upgrading costs for monitoring 
stations in the vicinity of large facilities. 








Recommendation 

























To enable the Ministry to cost-effectively increase the scope of its 
environmental oversight role, it should consider imposing conditions on 
certificates of approval that include: 


¢ greater use of self-reporting requirements that demonstrate owner 
) compliance, including, where warranted and practical, certification by 
independent experts; and 


* owners paying a greater share of costs associated with monitoring their 
compliance. 


Ministry Response 


The Division is committed to expanding the use of facility self-monitoring and 
compliance reporting conditions in certificates of approval. The Ministry will 
be developing guidelines to identify appropriate activities, circumstances and 
mechanisms to require third-party compliance verification by independent 
experts and reporting. 


The Ministry has recently taken action to put in place new self-monitoring and 
compliance reporting for municipal water treatment plants that directly 
addresses recommendations in the Provincial Auditor’s report regarding the 
costs of monitoring compliance. In the new drinking water protection 
regulation, third-party assessment in the form of a detailed engineering self- 
assessment report is required for all water treatment facilities within the next 
year. These engineering reports will be reviewed by ministry staff and new 
certificates of approval issued to ensure that adequate treatment equipment, 
and operating, self-monitoring and public reporting systems will be in place 
at all municipal water facilities. 
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Another example is the pilot program for new site-wide air certificates of 
approval that was initiated this past year. This pilot program has received 
considerable support, and incorporates conditions for self-monitoring and 
reporting, requiring owners to demonstrate compliance on an ongoing basis. 
Approximately 30 facilities per year are subject to this program. The site-wide 
pilot program will be monitored over time and rolled into a permanent 
program if results are satisfactory. As well, the Ministry is systematically 
requiring the larger industrial emitters to perform an engineering assessment 
of their compliance with conditions in certificates of approval and in 
regulations. 

















ENFORCEMENT 


The Environmental Protection Act is the Ministry’s primary piece of legislation providing 
powers and duties to environmental officers for abatement activities. Abatement activities 
include measures to control, prevent, reduce or eliminate pollution sources, including 
enforcement. Depending on the severity and circumstances of an incident, environmental 
officers can seek compliance either through voluntary cooperation or by using enforcement 
tools available under the legislation to compel corrective action. Measures for enforcing 
compliance include issuing control orders to individuals or companies and revoking or 
suspending certificates of approval. The Ministry may also take corrective action on its own in 
more serious cases where compliance is not obtained in a timely manner. 


Over 20,000 pollution occurrences are reported to the Ministry each year. Environmental 
officers respond to each according to established protocols. In addition, during 1999/2000, 
environmental officers conducted approximately 4,400 ministry-initiated inspections, including of 
water and sewage treatment plants, PCB storage sites, hazardous and non-hazardous waste 
landfill sites and transfer and recycling sites and facilities. 


Each year, approximately 1,000 formal investigations are initiated as a result of abatement 
activities. Penalties can include issuing summary convictions (fines) for lesser offences or 
seeking larger penalties as specified in the Act for significant violations. About 250 significant 
violations are referred annually to the Ministry of the Attorney General for prosecution. 


INSPECTION COVERAGE 


Inspections are an important means of assessing a facility’s level of compliance with legislative 
requirements and play a key role in promoting voluntary compliance. The visible presence of 
inspectors also helps promote public confidence that environmental standards are being 
enforced. While other sources of information, such as public complaints, are typically received 
after a significant pollution incident has occurred, inspections can be proactive in preventing 
such occurrences. 


Each year, the Ministry provides guidelines for ministry-initiated inspection activities to district 
and area offices. The guidelines outline mandatory inspections for certain types of facilities on a 
yearly or multi-year basis, set criteria for inspecting a minimum number of certain types of 
other facilities during the year, and provide criteria for selecting facilities that are optional, at the 
discretion of local staff. 
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The Ministry had a well-defined process in place for allocating available staff resources to 
ensure that the types of facilities inspected were based on priorities of highest risk. However, 
we noted that there had been a significant reduction in ministry-initiated inspections since 1996. 
While regional staff was reduced by over 25% during this period, ministry-initiated inspections 
decreased by 34%—trends that are illustrated in the chart below. For example, from 1995/96 to 
1999/2000, ministry-initiated inspections of hazardous and liquid industrial waste sites declined 
from about 2,000 to 1,190 per year. Similarly, inspections of municipal water treatment plants 
declined by over half, from over 400 to about 190 per year, over the past five years. 


Number of Inspections and Regional Staff 
(including Investigations and Enforcement) 





8,000 










== Average No. of Regional Staff (including Investigations and 
Enforcement) 


wx NO. of Ministry-initiated Inspections 









+ 7,000 





+ 6,000 


Staff 
suonsedsuj 


+ 5,000 


+ 4,000 





+ 3,000 





1995/96 1996/97 1997/98 1998/99 1999/2000 
Source: Ministry of the Environment data 


The Ministry has only been tracking compliance rates from inspection activities since 1998/99. 
For 1999/2000, it identified significant violations in 31% of the ministry-initiated inspections 
conducted. In view of the level of significant violations being detected and the large number of 
new approvals being issued each year, the Ministry ought to consider expanding its inspection 
activities. This would enable it to more effectively meet its legislative responsibilities. 








Recommendation 


To more effectively enforce compliance with environmental Cedasbee pal 
Ministry should explore options and develop procedures for significantly 
increasing its inspection coverage. 
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Ministry Response 


The Ministry will strategically target its compliance inspection and 
enforcement activities, while maintaining a cyclic baseline inspection 
function. 





RESOLVING VIOLATIONS 


The Ministry’s compliance guidelines require environmental officers to promptly assess the 
significance of any violation to determine whether an emergency situation exists that poses a 
potential or immediate danger to health or property. For all other situations of non-compliance, 
criteria exist for assessing whether voluntary or mandatory abatement activities are necessary. 


For environmental legislation to be effective, the Ministry needs to be taking enforcement action 
in an aggressive, appropriate and timely manner when violations are identified, particularly 
repeat violations. Our audit concluded that more stringent enforcement is required. Specifically, 
we noted instances where environmental officers: 


* — did not follow up on violations to ensure that the facility operator had subsequently 
corrected the deficiency; and 


* — responded inappropriately, such as using voluntary compliance measures where mandatory 
compliance was required or not following up in a timely manner. 


Ministry guidelines allow environmental officers the option to use voluntary instead of 
mandatory compliance measures if the reasons for the decision are documented in an 
occurrence report. We noted that voluntary compliance measures were generally used but such 
decisions were rarely documented. In this regard, there was frequently a lack of supervisory 
reviews of enforcement actions taken to ensure the appropriateness of decisions made by 
environmental officers. 


We were concerned that the guidelines allowed environmental officers the discretion to use 
voluntary measures even in cases of significant or repeat violations and in cases where 
corrective action had not been taken on a timely basis. 


During 1999, the Ministry conducted an internal review of the effectiveness of its inspection 
program. The internal review identified concerns similar to ours with regard to the inappropriate 
use of voluntary compliance measures. The internal review determined that in 69 of the 100 
inspection reports reviewed, violations were identified, including 22 considered significant by the 
Ministry. However, enforcement actions taken included only one control order issued and no 
fines or charges. In 19 cases, the environmental officer requested that the facility operator 
provide a voluntary abatement action plan; however, only one plan was actually received. 


In addition, the internal assessment noted that approximately one third of all violations identified 
were repeat violations. We noted that the policy of other regulatory programs is to prosecute if 
a violation found during a routine inspection has been identified on previous inspections. 
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Recommendation 


To make enforcement actions more timely and effective, the Ministry needs 
to strengthen its enforcement activities by: 


* taking appropriate action on violations and following up on a more timely 
_ basis; and 
* ensuring policies and procedures manuals encourage the use of more 
stringent compliance measures, where appropriate. 


Ministry Response 


The Ministry has made and continues to make significant changes in the way 
we respond to violations. The recent Program Effectiveness Review of our 
inspection program identified the need to clarify and reinforce the use of 
mandatory compliance measures. 


| In March 2000, the Division issued a clear direction to field staff to pursue 
more aggressive use of mandatory abatement actions against violators. 
Mandatory abatement includes actions such as issuing field orders that 
specify actions and completion dates to bring about compliance. The number 
of field orders issued since March 2000 has increased from an average of 20 
per month to a current average of 90 per month. 


Training is underway for all Provincial Officers in the use of the new 
enforcement tools provided under Bill 82. The investigator training is 
completed and the abatement officer training is underway. 


A new regulation has been drafted to provide for the use of Administrative 
Monetary Penalties (AMPs). This regulation is scheduled to be posted on the 
Environmental Bill of Rights Web site for public comment in September 2000. 
AMPs will allow for a penalty for violations to be levied against a violator that 
will not be processed in the court system. 








SIGNIFICANT VERSUS MINOR VIOLATIONS 


Ministry guidelines require environmental officers to focus their efforts in areas where the 
greatest environmental and human health benefit can be achieved. We found that Ministry 
management and staff only considered violations as significant where an adverse effect, such 
as a spill, was evident. Violations considered as minor included preventative measures outlined 
in environmental legislation, even though such violations, if not corrected, may increase the risk 
of extensive damages to the environment and human health. 


For example, the Ministry’s assessment of its inspection program conducted during 1999 
considered as minor 51 of the 58 violation types noted. Violations considered to be minor 
included failure to take or report samples of effluent or water quality, use of an uncertified 
operator, lack of a contingency plan should systems fail, and the operation of water and sewage 
facilities not in accordance with approval specifications. However, depending on the 
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circumstances, the violations identified could be significant, such as if the facilities were high 
risk and/or the operators had a past history of violations. 





Recommendation 


To minimize environmental and health risks, the Ministry should: 


* reassess its policies, procedures and criteria for determining the severity 
of violations; and 


* ensure that the significance of preventative measures is better 
understood and communicated to staff. 


Ministry Response 


The Ministry recognizes the challenge to prevent and to deal with risks and 
violations to environmental and public health. The Ministry is committed to 
directing its efforts to promoting preventative measures, ensuring 
compliance and enforcing the law. 


It recognizes that its traditional reliance on voluntary abatement is not 
achieving desired compliance levels and has shifted its emphasis to greater 
use of mandatory compliance tools to address violations. The impact of this 
shift may be noted in the increased number of orders being issued. This is 
particularly evident in the numerous orders being issued during the 
inspection blitz of all municipal water treatment facilities. 


Operations Division will complete a review of its operating policies and 
procedures this winter to ensure consistent and appropriate use of its 
compliance tools across all program areas. 


This Ministry has and continues to play an active part in the current 
government-wide review of the inspections, investigations and enforcement 
functions across 13 ministries. The Division will draw, where appropriate, 
from the work of this review to improve its policies and procedures for 
inspection, investigation and enforcement functions. 














MANAGING INSPECTIONS 


Our visits to ministry district offices identified the following discrepancies in the management of 
inspection activities: 


« Management at only three of the six districts we visited maintained detailed reports on 
facilities planned for inspection, those actually completed and the results. One of these 
districts also noted whether the facility should be inspected again in the following year. 
However, two other districts only kept copies of inspection reports and did not compare 
them with planned inspections, while another district was unable to provide us with a list of 
inspections planned for or completed during the current or previous years. 
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* None of the six district offices maintained documentation on how the Ministry’s selection 
criteria had been applied to arrive at the final list of sites planned for inspection. 


* There was no consistency among district and area offices on whether their ministry- 
initiated inspections were conducted on a surprise basis or by appointment with facility 
operators. Guidelines provided to staff do not address this matter. Surprise inspections have 
significant advantages for identifying violations, as well as acting as a greater deterrent. 


Recommendation 







To make its inspection program more effective in supporting the 
enforcement of environmental legislation, the Ministry should: 





* review its policies, procedures and guidelines over ministry-initiated 
___ inspection activities to ensure that adequate record keeping and 
reporting requirements are in place; and 


* ensure that inspections are consistently planned for and conducted. 
Ministry Response 


To ensure adequate record keeping and reporting, the Ministry has now 
implemented the first of its inspection databases, the Interim Inspection 
System for water treatment facilities. This system is part of the Ministry’s 
Environet information management strategy. It allows the Ministry to track 
the progress of inspections, to record findings and to follow up on 
deficiencies, as well as to generate inspection reports. Systems comparable 
to the Interim Inspection System are under development for all facilities 
inspected by ministry staff. 


Operations Division is reviewing its work planning manual and will 
strengthen the procedures for planned inspections. 








MANAGEMENT INFORMATION 


For enforcement action to be effective, violations have to be identified and resolved based on 
good information. In this regard, a number of our observations in this report were the result of 
i the Division’s staff not having appropriate systems and/or information to perform their duties 
more effectively and efficiently. We noted the following: 


* Upon approval, a copy of the certificate of approval is forwarded to the ministry district 
office for monitoring. However, the Ministry did not have an adequate tracking system to 
ensure that conditions of the approval were complied with. As a result, the district offices 
did not have the information needed to initiate follow-up action, such as sending reminder 
notices or conducting inspections, to enforce the conditions of approval in a timely manner. 





¢ The Ministry’s Occurrence Reporting Information System maintains information on Gees 
of public complaints and potential violations identified from inspections. Information to ae 
the generation, transportation and disposal of all hazardous and liquid industrial wastes is 
kept in the Hazardous Waste Information System. In addition, each district and area office 
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maintains its own list of sites that require mandatory cyclical inspections by using local 
databases or manually prepared lists. 


These systems only provided information on individual sites or facilities and did not allow 
for the creation of facility profiles or the linking of information between districts. Such 
capabilities would allow staff to better manage their activities and identify the types of 
environmental risks that exist in specific areas. For example, knowing all the approvals 
granted to facilities in the same industry or area would help identify those that had been 
operating without proper approvals. 


* The Occurrence Reporting Information System was not complete. We noted several 
examples of reports prepared by environmental officers that did not include coding of the 
violation type, which would allow for statistical reporting. The Ministry’s internal review of 
inspection activities also noted that violation types were recorded for only 32% of cases. 


* The Ministry had not conducted any detailed assessments of the types and frequency of 
violations typically identified from reports of pollution incidents and inspection activities. The 
information would be useful in identifying common difficulties that facilities are having in 
meeting requirements and in targeting enforcement efforts and educational awareness 
campaigns. 


The effectiveness of the Ministry’s compliance measures was also not assessed. For 
example, it would be useful to assess the extent that voluntary compliance measures, such 
as warnings and follow-ups, were effective and an efficient use of staff resources. In 
addition, the use of more immediate measures, such as fines, could be assessed to 
determine their success as a deterrent for violators. 
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Recommendation 


To better support and to improve the delivery of its enforcement efforts in 
protecting the environment, the Ministry should: 


* establish a system to identify all conditions of approvals that require 
follow-up by specific dates to assess owner compliance and where 
necessary, initiate timely enforcement action; 


* develop an accurate and comprehensive management information system 
to assist in identifying and prioritizing facilities for inspection; and 


* periodically assess the types and frequency of violations and the 
effectiveness of enforcement measures used. 





Ministry Response 





Operations Division agrees that a good information base is needed on which 
to make management operational decisions as well as set enforcement and 
inspection priorities. Our new Integrated Divisional System (IDS) has been 
designed with these goals in mind. Approvals are now generated on one 
system, which has the capability of alerting due dates for all conditions of a 
certificate of approval. 
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The design of the remainder of the IDS is almost completed. This next phase 
is scheduled for start-up in spring 2001. This will provide our Division with 
the new ability to run reports focused on facility type, inspection reports, 
violation types and enforcement actions. With this information we will have 
the capability of prioritizing facilities and sectors for inspections. This will 
enable the Division to strategically target its compliance inspection and 
enforcement activities, while maintaining a cyclic baseline inspection 
function. 


IDS will allow regional and district office management to closely monitor 
activities and focus field staff efforts into areas that have the most impact on 
environmental protection. This advancement is critical in moving from a 

| reactive to proactive approach in environmental compliance and protection. 


An Investigations and Enforcement Branch data system has just been 
enhanced to measure the level of enforcement action taken and sort by 
environmental program areas. This new information allows comparison of 
our efforts to specific areas of environmental enforcement. This information 
can be used to determine our effectiveness in sectors of environmental 
protection. 


An examination of compliance techniques and risk assessment across the 
various ministries is underway. The findings from this initiative will have a 
twofold result. They will provide feedback on the current tools in use by this 
Ministry and will provide further techniques that may have application across | 
the Ontario government or within specific ministries. As well, Operations 
Division will complete a review of its operating policies and procedures this 
winter to ensure consistent and appropriate use of its compliance tools 
across all program areas. 





UNPAID FINES 


Over the last five years, the total environmental fines imposed on violators by the justice system 
have averaged $1.5 million per year. However, as of March 31, 2000, the Ministry of the 
Attorney General reported that over $10 million in environmental fines that had accumulated 
over many years remained unpaid. The significant amount of unpaid fines compromises the 
extent to which enforcement measures act as an effective deterrent. 


The Ministry of the Environment has statutory authority to suspend certificates of approval for 
violators with outstanding fines. However, such suspensions are rarely imposed. Our analysis . 
identified a number of companies with outstanding fines that had active certificates. In view of 
the significant amount of outstanding fines, a system that issues warnings and, if necessary, 
suspends approvals would be an effective tool for enforcing payment of fines. 
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Recommendation 


To be more effective in enforcing environmental legislation and to improve 
collection of outstanding fines, the Ministry should investigate the reasons 
why outstanding fines are unpaid and use its statutory authority to suspend 
environmental approvals for violators who do not pay their fines. 


Ministry Response 





The responsibility for the collection of fines rests with the Ministry of the 
Attorney General. Notwithstanding the above, we support the Provincial 
Auditor’s recommendation and will use all the tools at our disposal to 
support the collection of fines as deterrence against environmental violators. 


Training of ministry staff on the new streamlined process to suspend all 
ministry licences to encourage payment of court fines will be completed by 
October 2000. Investigators have now been trained to initiate the forfeiture of 
seized property to the Crown from the defendant upon conviction or upon 
default of fine payment. 


The Ministry will initiate discussions with the Ministry of the Attorney General 
to provide the most efficient tracking of environmental fine payments to the 
courts. This will provide the information we need to then initiate appropriate 
licence suspensions and property forfeiture actions, to encourage fine 
payments. 

















CONTAMINATED SITES 


The Ministry generally directs its clean-up efforts to contaminated sites that are of significant 
public concern, such as when surrounding properties are adversely affected by the 
contamination. Owners or previous owners of those sites can be held legally and financially 
responsible for the contamination. 


The Ministry provides detailed guidelines to property owners and environmental consultants for 
assessing the environmental condition of a property, as well as for determining whether 
restoration 1s required and the kind of restoration needed to allow continued use or change of 
use of the site. The owner may also be required to obtain municipal approvals for land-use 
changes where contamination impairs the use of the property. Site assessments are particularly 
important to potential purchasers since contamination could restrict the use of the site and may 
transfer the clean-up costs to the new owner. 


The Ministry is responsible for administering an Environmental Clean-Up Fund, which makes 
funding available for serious or urgent environmental problems. Funds are mainly directed for 
cleaning up or restoring contaminated sites where the responsible persons cannot be identified, 
are unable to pay or where enforcement efforts have not been effective. In many 
Environmental Clean-Up Fund cases, it is not practical or technically feasible to fully clean up 
the sites after prolonged or severe contamination. Nevertheless, in severe cases, the Ministry 
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has spent significant funds to clean up such sites in order to contain the damage or minimize the 
environmental and health risks associated with the contamination. 


For the 1999/2000 fiscal year, the Fund provided funding to over 45 clean-up projects totalling 
$5 million. Since it was established in 1985, the Fund has paid out approximately $160 million. 


IDENTIFYING CONTAMINATED SITES 


Early identification of contaminated sites can minimize damage to the environment, reduce 
costs and identify the responsible parties. The Ministry can then require corporations to 
establish funds to clean up their site while they are still operating. 


The Ministry did not have a program to identify contaminated sites nor a central inventory of 
contaminated sites. As a result, no timetable or cost estimates for clean-up of contaminated 
sites had been established. Priorities were set reactively when contaminated sites were 
identified from pollution incident reports. 


Specifically: 


* The Ministry did not have a system that required owners of high-risk industrial and 
commercial sites to conduct site assessments to determine the extent of contamination on 
their sites. Site assessments were only required by the Ministry if it was known or 
suspected that contaminants were causing off-site, adverse effects. In addition, the 
Ministry did not routinely conduct assessments of remediation programs implemented by 
corporations to ensure that their efforts and progress were satisfactory. 


¢ In 1991, we reported that the Ministry had not taken action to assess closed private and 
municipal landfill sites to determine whether they posed a hazard to surrounding areas. 
These sites typically were built and operated prior to ministry regulations and may have 
lacked controls to prevent leaching of contaminants to off-site areas. At that time, the 
Ministry had classified 700 of the 2,400 landfill sites as a priority, of which 200 had received 
an inspection and another 250 an administrative review. We noted that no further action had 
been taken since 1991. 


¢ No program or initiative existed for identifying abandoned, underground, fuel storage tanks, 
which represented about half of the projects being funded by the Environmental Clean-Up 
Fund. 


We also noted that, under the Ministry’s Provincial Water Protection Fund, $5 million had been 
made available to municipalities since 1997 to identify potential contamination of groundwater 
supplies. However, as of March 31, 2000, the funds had been fully allocated to 39 municipalities 
and no further funds were available for this purpose. 


LIABILITY AND FINANCIAL CONSIDERATIONS 


Several U.S. and Canadian jurisdictions have legislation that limits responsibility for owners who 
have complied with clean-up guidelines and for parties not directly responsible for causing a 
contamination. In Ontario, there is no legislation that requires the Ministry to provide owners 
who have restored contaminated sites with assurance that the restoration was performed in 
compliance with its guidelines. For example, if the Ministry's standards change after clean-up, 
owners want assurance that additional clean-up will not be required. Without such assurances, 
owners may be reluctant to voluntarily clean-up sites. 
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We also noted that other jurisdictions have programs to promote redevelopment of brownfield 
sites. These are typically large, abandoned or decaying industrial or commercial sites with 
contamination that limits their future use. The costs to clean up these sites are high. According 
to the Ministry, Ontario does not have as large a problem as does the U.S. However, it 
acknowledges that brownfield areas do exist, but it does not keep an inventory of these areas. 


During our audit, the Ministry was in the process of assessing liability issues and the need for 
brownfield redevelopment programs. 
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Recommendation 


To help ensure that the Ministry’s efforts and programs are effective in 
minimizing damage to the environment, the Ministry should develop a 
strategy for: 


* the early identification of all contaminated sites and responsible parties, 
which would allow for prioritizing clean-up requirements; and 


* the establishment of incentives to encourage property owners to 
voluntarily clean up contaminated sites. 


Ministry Response 





We agree in principle that early identification of contaminated sites and 
responsible parties and that finding ways to ensure owners clean up those 
sites are key factors in protecting the environment. 


Our efforts are focused on these priorities: 


* working jointly with the ministries of Municipal Affairs and Housing and 
Economic Development and Trade on a brownfield contaminated sites 
policy review; 

* implementing the Memorandum of Understanding with the Technical 
Standards and Safety Authority to deal with leaking underground fuel 
storage tanks; 

* monitoring corrective actions of municipalities dealing with closed landfill 
sites; and 

* providing technical advice to the Ministry of Northern Development and 
Mines on implementing the new (April 2000) clean-up program of old/ 
abandoned mine sites. 








The Ministry will consider developing a strategy, with the cooperative 
involvement of other levels of government, to expand efforts that would 
encompass a wider range of contaminated sites and address means of 
identifying those sites earlier and encouraging clean-up of those sites. 
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INFORMATION SYSTEM CONTRACT 


During our audit, the Ministry was in the process of designing and developing a new 
computerized Integrated Divisional System (IDS). When fully implemented, the new system is 
expected to improve the informational needs and efficiency of the Division by enabling key 
environmental data to be shared between branches and regional and district offices. The 
system would also replace the many separate systems that are in place, including the ones used 
in processing new approvals and monitoring and tracking pollution incident reports. 


At the time of our audit, we noted that the IDS project was delayed and was experiencing cost 
overruns. 


In July 1997, the Ministry awarded a contract to develop the Integrated Divisional System to a 
successful bidder for approximately $1.5 million. The project, including business process 
redesign, conversion of historical data, software development, and software and hardware 
purchases, was to be completed by July 1999. At the time, the Ministry did not seek 
Management Board approval, which was required for information technology projects over 

$1 million, since it only considered the hardware and software costs of approximately $300,000 
in the information technology component of the project. 


In December 1999, Management Board directed the Ministry to obtain approval for all costs 
associated with the project. In this regard, on February 15, 2000, Management Board approved 
a total cost of about $2.9 million for this project, which included actual and estimated costs from 
the 1997/98 fiscal year to the 2002/03 fiscal year. 


As of April 2000, the hardware and software had been purchased and certificates of approval 
since November 1999 had been processed using the new system. Payments to the contractor 
totalled approximately $700,000 plus an additional $800,000 for enhancements not covered by 
the original contract. However, existing records had not been converted to the new system, the 
information was not available outside of the Approvals Unit, and the system had not been 
designed for the rest of the activities in the Division. 


During our audit, the Division was in the process of renegotiating its agreement with the 
contractor, including the completion dates, clarification of the original scope of the project, and 
any further enhancements needed to the original design. Further cost escalations, if any, would 
require additional approval by Management Board. 





Recommendation 


To facilitate the efficient and effective delivery of the Division’s programs, 
the Ministry should ensure that the remaining portions of the Integrated 
Divisional System are completed in a timely and cost-effective manner and in 


accordance with required approvals. 


Ministry Response 


At the time of the audit, the Integrated Divisional System (IDS) was 
experiencing delays and cost overruns due to a vendor/sub-contractor 
dispute. Negotiations were taking place regarding what the vendor agreed to 
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provide and the expectations of the Operations Division development team. A 
series of documents were developed and agreed to by both parties and this 
project is now back on track. The vendor will deliver the system for the 
original price quoted by January 2001. An agreement was also reached 
whereby the cost of any change orders for improvements to IDS would 
remain at the original per-diem rate. 







Management Board approval to proceed with this project has also been 
obtained. Any extra cost to the IDS project will either be allocated from within 
the Ministry or a new Management Board submission will be made. 











MEASURING AND REPORTING ON PROGRAM 
EFFECTIVENESS 


STATE OF THE ENVIRONMENT REPORTING 


The Ministry’s mandate is to protect the natural environment and encourage conservation of 
water, energy and material resources. Its vision is an Ontario where clean air, water and land 
sustain human health, recreation, commerce and industry. The Operations Division supports the 
Ministry’s mandate and vision. 


The Ministry reported individual measures of effectiveness through its annual Business Plan. 
However, the Business Plan did not provide a comprehensive assessment of the overall impact 
of the Ministry’s efforts on the environment. For example, under its goal for Cleaner Land, the 
only measure reported on publicly was the percentage of PCBs in storage that had been 
destroyed. Also, under its goal for Healthier Ecosystems, the efficiency of processing approvals 
and environmental assessments were measured but the outcomes of these approvals were not. 


The Ministry also periodically published reports and news releases on its activity over certain 
key environmental areas. While these documents tended to be informative of recent successes 
or initiatives taken by the Ministry in addressing specific environmental issues, they generally 
lacked objective appraisals of the quality of the environment. In addition, the myriad of ministry 
reports and information sources on the environment that were available did not allow for easy 
and effective evaluation of the overall state of the environment. 


At the time of our audit, the federal government, the U.S. Environmental Protection Agency, 
several U.S. states, and other provinces had developed or were in the process of developing 
indicators for measuring changes to the environment over time. A state of the environment 
report, as it is generally referred to, acts as a mechanism to help monitor progress towards 
achieving environmental goals, targets and quality. Such a report could also be used to 
objectively report on Ontario’s progress in meeting established provincial, national and 
international environmental standards and commitments for pollution reduction targets and 
clean-up efforts. 
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Recommendation 


To ensure that the Ministry’s progress in managing the environment is 
measured and communicated in an objective manner, a state of the 
environment report should periodically be prepared using a set of 
comprehensive outcome measures that assess the quality of and changes to | 
the environment over time. 


Ministry Response 


The Ministry recognizes its role to provide the public with information on how | 
it is progressing with its programs. It is equally important that the Ministry | 
facilitate the sharing of information on how other sectors of society are 
performing in their responsibilities relating to the environment. 


The Ministry is committed to continually improving its performance 
measures in its Business Plans. The current Business Plan contains more 
outcome-based measures than in previous years by which to gauge progress 
over time. 


The Ministry is increasing the use of its Web page to report on: 


¢ non-compliant facilities discharging to water; 

¢ deficiencies identified by inspections of municipal water treatment plants; 
and 

* new regulations, one for water treatment plants and another for facilities 
with air emissions, that require the reporting of sampling and monitoring 
results. 


The Ministry has also, in partnership with conservation authorities, launched 


a groundwater monitoring network. This will provide baseline information on 
the state of the groundwater. 


The Ministry has approved and is launching its major initiative, Environet, to 
make information available to the public. The goal is to provide the public 
with access to all public information within the Ministry, through one single 
Internet site. This will greatly enhance and speed up the provision of vast 
amounts of information to the public in a user-friendly format. 


The first three projects of this initiative are: 

* water monitoring and compliance information from water treatment 
plants; 

* hazardous waste information on generators and receivers; and 

* anair emission inventory—for facilities with regulated air emissions. 

In addition, those responsible for emitting any pollutant will have to monitor 


and report publicly on their performance. All of this information will 
eventually be required to be posted on the Web on a continual basis. 


These projects are to be in place by the summer of 2001. 
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DIVISIONAL PERFORMANCE REPORTING 


The Division reported internally on the performance of its key areas: the Approvals Unit, 
abatement activities by each region, and the Investigations and Enforcement Branch. Its 
performance measures were largely based on activities planned for and completed but few 
outcomes or results: 


* The Approvals Unit measured and reported on the efficiency of its operations, such as the 
average turnaround time and number of approvals issued. However, there were no 
indicators in place to assess the quality of the over 220,000 approvals issued. Measures that 
could be considered include: the average age of approvals, whether they are up-to-date 
with current standards and the extent to which approvals incorporate self-monitoring and 
reporting requirements. 


* Each region monitored the number of ministry-initiated inspections completed. A key 
measure in the Ministry’s Business Plan was the inspection pass rate for municipal water 
and sewage treatment facilities. Other measures being piloted were the overall pass rate 
for all ministry-initiated inspections and the percentage of environmental problems resolved. 
To better monitor compliance efforts by each district, a new monthly report was 
implemented in mid-1999 summarizing pollution incident reports and enforcement measures 
used. 


Other key measures could be rates of non-compliance by each major industry sector, 

repeat violations noted within two years of previous enforcement, and the average number 
of days to bring a facility into compliance. In addition, measures could be established for the 
Ministry’s activities covering contaminated sites. 


* The Investigations and Enforcement Branch measured its results primarily on the number 
of convictions obtained and fines imposed. The Branch was also planning to pilot an 
efficiency measure that would assess staff time to complete investigations. A number of 
other measures that would be appropriate include convictions of repeat offenders, amount 
of unpaid fines and the number of cases prosecuted by investigators that did not require the 
assistance of the Ministry of the Attorney General. In addition, the activities of the new 
Smog Patrol should be reported on separately since the unit operates independently. 
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Recommendation 

To provide a more comprehensive assessment of the Division’s contribution 
to protecting the environment, the Ministry should develop more results- 
oriented performance indicators to measure and report on the effectiveness 
of the Division’s operations. 








Ministry Response 


The Ministry is committed to continuously improving performance measures 
reported in the public Business Plan and internal, program-level performance 
reporting. The Division will continue to develop measures that focus on 
effectiveness indicators that are results-oriented and/or demonstrate the 
value added by the regulatory processes to environmental protection. 
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In the area of enforcement, the Business Plan includes measures for the 
enhanced use of environmental enforcement tools to increase compliance 
and environmental protection. These measures reflect a commitment to 
utilize the improved investigation and enforcement tools contained in Bill 82. 









At the same time, program-level measures for the Ministry’s major program 
areas are also being examined and improved upon. Significant action has 
been taken to ensure the suite of performance measures tracked by the 
Ministry will provide critical information for improving the effectiveness of 
key ministry programs. The Ministry has also developed, or is in the process 
of developing, performance measures, which Operations Division is a partner 
in. These include the pilot for the Water Quality Index; the groundwater 
monitoring network; the assessment of water quality in lakes, rivers and 
tributaries; mandatory air emissions reporting; the drinking water 
compliance and drinking water surveillance programs; wastewater 
compliance reporting; Drive Clean performance; and the environmental 
results summarized in the Guide to Eating Sport Fish in Ontario. 
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MINISTRY OF FINANCE | 


Retail Sales Tax Program 





BACKGROUND 


Currently, the Retail Sales Tax Act imposes a general sales tax of 8% on the retail price of 
most goods and services sold to final consumers. The Act also levies sales tax at variable rates 
between 4% (for example, on automobile insurance) and 12% (for example, on alcoholic 
beverages). 


Sixty-three types of tax exemptions cover thousands of items. Many of these exemptions aim to 
reduce tax regression or have economic or social objectives. Examples of such exemptions 
include children’s clothing, equipment designed for use by people with disabilities and goods 
purchased by Status Indians under certain conditions. 


As at December 31, 1999, approximately 380,000 vendors were registered to collect and remit 
retail sales tax (RST) to the province (for vendor population by type, see table in section Audit 
Coverage and Selection). RST receipts for the 1999/2000 fiscal year totalled approximately 
$12.6 billion, net of $159.5 million in refunds, which represented 21% of the province’s total 
revenue. 


The chart below shows tax receipts for the last six years. 


Retail Sales Tax Receipts 
($ billions) 











1994/95 1995/96 1996/97 1997/98 1998/99 1999/2000 


Source: Ministry of Finance 
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The Retail Sales Tax Branch, operating through its head office in Oshawa and four regional and 
seven field offices throughout the province, has primary responsibility for the administration and 
enforcement of the Retail Sales Tax Act. Other ministry branches provide specialized support: 
the Special Investigations Branch investigates complex cases of tax evasion; the Collections 
and Compliance Branch collects overdue amounts, follows up overdue returns and identifies 
non-registered vendors; the Tax Appeals Branch handles all objections and appeals; and the 
Revenue Operations and Client Services Branch processes all tax returns and related 
payments. 


AUDIT OBJECTIVE AND SCOPE 


Our audit objective was to assess whether the Ministry had adequate procedures in place to 
ensure that the appropriate amount of RST was collected and remitted to the province on a 
timely basis, in accordance with statutory requirements. 


The scope of our audit work included a review and analysis of relevant ministry files and 
administrative procedures as well as interviews with appropriate staff at the Ministry’s head 
office and three regional offices, the Special Investigations Branch, the Collections and 
Compliance Branch, the Tax Appeals Branch and the Revenue Operations and Client Services 
Branch. 


Prior to the commencement of our fieldwork, we identified the criteria we would use to address 
our audit objectives. These criteria were reviewed and agreed to by senior ministry 
management. 


Our audit work covered the period up to March 31, 2000, with emphasis on the procedures in 
place with respect to RST revenues processed in the 1999/2000 fiscal year. Our audit was 
conducted in accordance with the standards for assurance engagements encompassing value 
for money and compliance, established by the Canadian Institute of Chartered Accountants, 
and accordingly included such tests and other procedures as we considered necessary in the 
circumstances. 


We reviewed and relied on the work performed annually on the processing of RST returns at 
the Revenue Operations and Client Services Branch by the Ministry’s Audit Services Branch. 
However, the Branch’s work did not affect the other aspects of our audit because the Branch 
had not conducted work on these aspects of the Retail Sales Tax Program during the last three 
years. 


OVERALL AUDIT CONCLUSIONS 


Although RST revenues have gone up significantly and the Ministry has made improvements to 
its administration of this program since we last audited it in 1995, we concluded that the 
Ministry needed to further improve its procedures to ensure that: 


¢ All vendors selling taxable goods and services are registered with the Ministry to collect 
and remit the appropriate amount of tax. 
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* Overdue sales tax returns are followed up and received on a timely basis. 


* Collection actions for overdue amounts outstanding are strengthened and performed on a 
more timely basis. 


With respect to the Ministry’s enforcement initiatives, we concluded that the Ministry needed 
to: 


* — research the nature and extent of the underground economy in order to identify aspects of 
the economy in need of more rigorous service and enforcement action; 


* increase its overall audit coverage to the desired level established by the Ministry; and 


* more representatively select vendors for audit with a view not only to maximizing audit 
revenues but also encouraging broad-based voluntary compliance. 


We also noted that adequate controls were in place to ensure that sales tax returns were 
reviewed for accuracy and were correctly processed on a timely basis. 


DETAILED AUDIT OBSERVATIONS 


All vendors selling taxable goods and services must be registered with the Ministry and have 
available their RST permit at their place of business. In addition, they are required to file an 
RST return and remit the sales tax collected before the 23rd day following the end of their 
reporting period. Reporting periods vary from monthly to semi-annually depending on the 
amount of tax collected by a vendor. Sales tax returns and remittances may be mailed directly 
to the Ministry’s Revenue Operations and Client Services Branch, dropped off at any ministry 
office or paid at most financial institutions in Ontario. 


The Ministry’s objective is to administer the collection of RST in a way that will encourage 
broad-based, voluntary compliance, while at the same time enforcing compliance to maintain 
equity and public confidence in the fairness of the tax system. To achieve this objective, 
approximately 240 administrative staff work to promote voluntary compliance by providing 
support services to vendors, such as responding to general inquiries and keeping the tax roll up- 
to-date. Approximately 560 staff are involved in audit and refund-related activities. 


In our 1995 report on the RST Program, we recommended that audit coverage be significantly 
increased. In 1995, the Ministry employed 152 RST auditors; by 2000, the number of auditors 
had increased to 418. The chart below illustrates the resulting increase in RST assessments. 
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Retail Sales Tax Assessments by Year 
($ millions) 























1994/95 1995/96 1996/97 1997/98 1998/99 1999/2000 


Source: Ministry of Finance 


TAX ROLL 


Having a complete and accurate tax roll of vendors selling taxable goods and services is an 
essential first step in effectively administering the RST Program and ensuring that the correct 
amount of RST is remitted to the province. 


We reviewed the Ministry’s procedures for ensuring that the tax roll was complete and 
accurate. Our review resulted in the following concerns: 


¢ The Ministry did not have adequate procedures in place to ensure that the Ministry of 
Consumer and Commercial Relations notified the RST Branch of potential new vendors 
when new businesses were incorporated or otherwise registered with it. Instead, the 
Branch relied on new businesses to register voluntarily if they intended to sell taxable goods 
or services in Ontario. 


¢ The Ministry had no regular procedures for identifying existing non-registered businesses 
through data matches with other government databases. Although the Ministry had agreed 
to establish such procedures in response to a recommendation in our 1995 audit report on 
the RST Program, we were advised that data matching had not been implemented on a 
large scale due to Y2K priorities and a ministry freeze on changes to the existing computer 
system. 

¢ Similarly, the Ministry needed to strengthen its procedures for regularly identifying non- 
registered vendors at their places of business or at points of sale. 


Without a complete and accurate tax roll, the Ministry cannot ensure that the correct amount of 
RST is collected and remitted to the province. 
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Recommendation 





To help ensure that the tax roll for vendors that sell taxable goods and 
services is complete and accurate, the Ministry should: 


* work with the Ministry of Consumer and Commercial Relations to ensure 
that it is appropriately advised of newly registered businesses for 
potential inclusion in the tax roll; and 

* implement additional procedures needed for identifying non-registered 
vendors selling taxable goods and services through data matching with 
other government databases or at their places of business. 





Ministry Response 


Most businesses register voluntarily with the Retail Sales Tax (RST) Branch. 
The Ministry is currently taking steps to link data already provided by the 
Ministry of Consumer and Commercial Relations (MCCR) with the RST 
database. 


The Ministry already has an exchange with MCCR via the Ontario Business 
Connects (OBC) workstations. If a business registers via OBC, the Ministry is 
notified and the RST database is updated to record the new vendor. 


Although the Ministry has a number of data matching programs already in 
place, it agrees that it could do more in this area, including further contact 
with MCCR regarding data matching opportunities. 





The Ministry has developed and implemented a process for identifying non- 
registered vendors by matching RST and GST databases. The Ministry is also 
developing a new work management system, which is scheduled to be 
implemented later this year, to assist with the investigation and registration of 
businesses identified as non-registered through the matching process. 








OVERDUE SALES TAX RETURNS 


All registered vendors must file a sales tax return no later than the 23rd day following the end 
of their reporting period whether or not they had any sales or collected any tax during that 
period. As at December 31, 1999, approximately 51,000 vendors had defaulted on filing the 
required 160,000 sales tax returns. 


Depending on the status of the account in default, the Ministry takes a number of progressive 

follow-up actions (including system-generated reminder notices followed by telephone contact), 
which in turn depend on the risk of loss. If the default continues, an estimated assessment is to 
be issued based on previous returns and remittances or remittances from other similar vendors. 


We reviewed a number of vendor defaults assigned to regional and head offices for follow-up 
and found that required follow-up work was not timely for approximately half of the assigned 
accounts we reviewed. For example: 
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* Three to twenty months elapsed after the assignment was made before an 


y follow-up 
action was taken. 


* Three to twelve months elapsed before the second phone call was made or an estimated 
assessment of taxes owed was issued. 


We understand that many of the initial delays were attributable to problems associated with 
integrating the RST System with the Integrated Collections System (ICS), where a large 
number of accounts did not get assigned for follow-up action between May 1998 and 
September 1999. 


Prompt follow-up of vendors in default of filing a return is important because, as the time period 


between defaults and follow-ups increases, the possibility of collecting outstanding remittances 
diminishes. 


















Recommendation 





To identify and follow up outstanding retail sales tax (RST) returns and | 
| required remittances more promptly, the Ministry should ensure that: 


* vendors with overdue tax returns are contacted within the required time 
periods; and : 
* when necessary, estimated assessments are issued on a more timely 
basis. — 
Ministry Response 
The Ministry experienced a system problem that resulted in a backlog of RST 
non-filer accounts being referred for follow-up action. The Ministry identified 
the system problem in June 1999 and resolved it immediately. The Ministry 


routinely prepares estimated assessments in failure-to-file situations as part 
of its telephone follow-up. 


The Ministry has recently expanded its resources to ensure more timely 
follow-up and assessment of outstanding sales tax returns and is also 
developing a new work management system to assist with the collection of 
past due returns. This system is scheduled to be implemented later this year. 





4 





COLLECTIONS 


Accounts receivable result primarily from RST assessments and from vendors submitting RST 
returns without full payment. By October 31, 1999, RST accounts receivable had increased to 
$587 million (before an allowance for doubtful accounts of $201 million) from $272 million | 
(before an allowance for doubtful accounts of $96 million) at the time of our last audit of ue 
program in 1995. We understand that factors that contributed to this increase include a growth 
in revenue of $3.5 billion over this period and a significant increase in the amount of 
assessments resulting from audits. 
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Outstanding Retail Sales Tax Accounts by Amount 
as of October 31, 1999 


% of Total 
Accounts with Number of 
Outstanding Accounts with 
Balances of: Outstanding 
Balances 


Total Value 
($ millions) 


$100,000+ 

$25,001 to $100,000 
$5,001 to $25,000 
$1,000 to $5,000 
Less than $1,000 























Source: Ministry of Finance 


Outstanding Retail Sales Tax Amounts by Time Overdue 
as of October 31, 1999 


Dollar - 
Past Due Amount alias 
($ millions) 
ute 


0 - 90 days 
90 days to one year 20 











One year to two years Le 
Two years to three years 
More than three years 














Source: Ministry of Finance 


When an account becomes overdue, the Ministry’s RST System normally generates two 
reminder letters. If the account remains outstanding, the Integrated Collection System assigns it 
to a collector in either the Ministry’s head office or a regional office for further follow-up. 
Within 30 days of the assignment, the collector is expected to send additional collection letters, 
conduct follow-up telephone calls and consider initiating legal actions, such as bank 
garnishments and liens. 


Large outstanding balances are generally assigned to the individual “work stacks” of senior 
collectors while smaller balances are assigned to the work stacks of other collectors. In 
addition, the Integrated Collection System has a number of specific work stacks for accounts 
identified as high-risk businesses. 
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All collectors are expected to address the accounts in their work stacks in descending order of 
outstanding balance amounts. For example, before working on a particular account, the 
collector must first work on all the accounts with higher outstanding balances than that one. 


We reviewed a sample of accounts receivable files with outstanding balances over $100,000 
and found that the required collection efforts had been taken on these accounts on a timely 
basis. However, for accounts with outstanding balances less than $100,000, we found that 
collection efforts were not timely for approximately 20% of the files we reviewed, many of 
which were under $10,000. For example, many of these accounts had not been followed up for 
periods of up to two years. 





Recommendation 


To help maximize the collection of all outstanding accounts receivable, the 
Ministry should ensure that all accounts receive timely action by a collector, 
including those under $10,000. 


‘Ministry Response _ 


We agree that all outstanding accounts should receive timely follow-up. The 
Ministry allocates all its resources on a priority basis to ensure accounts with 
the greatest risk of loss receive timely action. The majority of outstanding 
accounts identified by the Provincial Auditor as requiring more timely action 

were considered by the Ministry to be unlikely to be collected. 





The Ministry is currently improving management information processes for 
monitoring accounts receivable to ensure timely action is taken. 








ENFORCEMENT 
TAX GAP 


The “tax gap” refers to the amount of RST that is due to the province but never gets remitted. 
The tax gap results primarily from both registered and unregistered vendors that either do not 
charge tax on taxable sales or that charge tax but do not remit it to the province. 


Although the nature and amount of the tax gap is difficult to quantify, it is generally thought to 
be significant and attributable to the underground economy. Therefore, the Ministry 1s focusing 
its resources on identifying the causes of the tax gap rather than on determining the amount. 


In our 1995 report, we recommended that the Ministry conduct research into the underground 
economy and use its results to focus its efforts on reducing the tax gap. In addition, the 
Standing Committee on Public Accounts, as a result of its hearings, recommended that research 
and analysis focus on non-compliance in order to ensure enforcement resources are used in the 
most cost-effective manner. 


The Ministry agreed with these recommendations in principle and committed to include the 


Pare ath inistration 
necessary management performance indicators 1n its planned Integrated Tax Administ 
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System, which was to have been completed in 1997. However, the completion of that system 
has been delayed, with the result that research into the underground economy has not been 
undertaken. 


While we acknowledge that identifying and estimating the tax gap is a difficult and imprecise 
task, there are initiatives that the Ministry could currently undertake to identify segments of the 
underground economy contributing to the tax gap. These include: 


* more emphasis on identifying non-registered vendors and analyzing the extent and impact 
of non-registered vendors by industry sector; and 


* analyzing the dollar value of assessments issued as a result of audits conducted by industry 
sector and type of vendor, relative to the number of audits completed for a particular sector. 


Both of these initiatives would help to identify aspects of the economy in need of more rigorous 
compliance and enforcement activities aimed at reducing the tax gap. 








Recommendation 


To help reduce the tax gap, the Ministry should conduct the research 
necessary to identify significant aspects of the underground economy and 
focus its compliance and enforcement efforts on these aspects. 


Ministry Response 





Ontario is taking steps to reduce taxation through tax cuts, which should 
reduce the tax gap. 


The Ministry is in the process of restructuring its identification and 
compliance functions, which includes forming a Compliance Planning and 
Research Unit. This unit will have primary responsibility in this area as well as 
a coordination role with other enforcement functions (especially audit). In 
addition, the Ministry is working with the Canada Customs and Revenue 
Agency (CCRA) and other provinces to leverage best practices in the area of 
compliance research. Ontario is committed to working with all the provinces 
and CCRA to address the underground economy and has taken a lead role in 
establishing a National Working Group on the Underground Economy, which 
had its inaugural meeting in June, 2000. 





The Ministry is currently reviewing the Retail Sales Tax Branch’s existing 
business processes and mainframe operating system (BASYS). This review 
will include investigating opportunities to improve compliance and reduce tax 
leakage by using intelligent systems to capture more information in order to 
enhance audit selection and allow for modelling and forecasting impacts of 
various initiatives. 
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AUDIT COVERAGE AND SELECTION 


AUDIT COVERAGE 


The objective of the Ministry’s audit efforts is twofold: to determine whether individual vendors 
selected for audit have remitted the correct amount of tax owed; and to encourage voluntary 
compliance in the broader vendor community. To meet these objectives it is important that an 
adequate and representative level of audit coverage is maintained. 


Vendors are grouped into one of three categories, as the following table illustrates. 


Vendor Population by Type 


Number of Vendors as | Annual Tax ss | 
of December 31,1999 | Annual Revenue — or Remitted | 
: 














Type 








Over $1 billion or Over $5 million 
Over $30 million or Over $100,000 
Under $30 million or Under $1 00,000 














Source: Ministry of Finance 


Although the audit coverage for small vendors has increased significantly since the time of our 
last audit in 1995, the Ministry has not included all segments of the small vendor population in its 
audit coverage. In addition, it is necessary that the Ministry review its overall audit coverage of 
small vendors to ensure visibility in the broader vendor community and thereby encourage 
broad-based voluntary compliance. 


As illustrated in the following table, the recovery rate per audit hour varies by vendor type. 


Per-Audit-Hour Recovery Rates by Type of Vendor 


Average Recovery Per | 
Vendor Type Audit Hour for 2000 
Large $865 


Medium 
Small ) $290 


Overall Average Recovery per Audit Hour $437 


Source: Ministry of Finance 






























We noted that, overall, 70% of all audits resulted in an assessment of taxes owed. In addition, 
despite the relatively significant increase in the audit coverage of small vendors, where there 1s 
a lower tax recovery per audit hour than for larger vendors, the current overall average — 
recovery per audit hour of $437 is comparable to the average recovery of $466 per hour in 


1995: 
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Good audit coverage is the cornerstone of tax enforcement. Without it, the Ministry cannot 
effectively detect whether taxes are being remitted as required, and broad-based voluntary 
compliance by the vendor community is not encouraged. 





Recommendation 


To detect unpaid retail sales taxes owed to the province and to encourage 
broad-based voluntary compliance, the Ministry should include all segments 
of the small vendor population in its audit coverage and attain overall audit 
coverage levels that will ensure visibility in the broader vendor community. 


Ministry Response 


Since 1995, the Ministry has increased its retail sales tax audit coverage of 
small vendors by 183%. The Ministry will undertake a review of its audit 
coverage levels of small vendors to determine the impact of broadening it to 
include all segments of the small vendor population. Any expansion would 
take into consideration the resource impacts and would be balanced against 
the Ministry’s ability to maintain a large and visible impact of its established 
audit coverage. 

















AUDIT SELECTION 


To facilitate the process of selecting vendors for audit, like vendors are grouped into one of 150 
profile codes. Among other things, these groupings enable the Ministry to compare tax yields of 
all vendors in a group and generally ensure that some vendors are selected for audit from a 
broad spectrum of the tax roll. 


Our review of audits completed during the 1999/2000 fiscal year by profile codes containing at 
least 500 vendors indicated that for 43% of them audit coverage was minimal. These codes 
included what we considered high-risk vendors. In addition, only 13 profile codes were audited 
to the overall desired level. 


Ministry staff advised us that they selected vendors for audit based on their judgment, with an 
emphasis on potential audit recoveries rather than representative selection. Although this 
approach meets one part of the Ministry’s audit objective, to determine whether individual 
vendors have remitted the correct amount of tax owed, it provides no assurance that audit 
coverage is sufficiently representative to encourage voluntary compliance in the broader vendor 
community. 


Our review of the audit selection process also indicated that: 


* for a sample of vendor files we reviewed, the assigned profile code did not agree with the 
vendor’s registration form approximately 25% of the time; and 


* at one regional tax office, audit managers selected vendors for potential audit by placing 
them in their personal audit banks, which prevented other auditors from auditing them. 
Approximately one-third of these vendors had been in audit banks for over one year, with 
an average time of 2.5 years, without any audit work being performed. 
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In addition, at two of the three regional tax offices we visited, some audits had been in progress 
. . . © 

for an extended period of time, averaging about 1.7 years. Ministry staff were unable to provide 

reasonable explanations for the delays in about one quarter of these instances. 


If audit selection does not sufficiently reflect the full spectrum of vendors or if audits are not 
performed in a timely manner, vendor confidence in the fairness and effectiveness of the RST 
Program may weaken which, over time, could result in lower levels of compliance. 





— 















Recommendation 


To help ensure that it meets its objectives for encouraging voluntary vendor | 
compliance as well as for recovery of retail sales taxes owing, the Ministry 
should: 


e select vendor files for audit on a more representative basis, with a view 
to encouraging broad-based, voluntary compliance; and 


# ensure that selected audits commence and are completed on a timely 
basis. 


Ministry Response 


Experience has demonstrated that certain profile codes represent greater 
compliance risk than others. To ensure the effective utilization of audit 
resources and enhance compliance, the Ministry employs a risk-based 
approach to audit selection. The Ministry intends to enhance its audit 
selection process using external data sources to detect non-compliance. Any 
expansion to other areas would have to take into consideration the resource 
implications and be balanced against the Retail Sales Tax Branch’s ability to 
maintain a large and visible impact in the sectors now selected. 


We agree that audits should commence and be completed on a timely basis. 
Existing processes will be changed, where necessary, and audits will be 
reviewed by quality assurance staff on a periodic basis to ensure audits are 
assigned and completed on a timely basis. 








MANAGEMENT INFORMATION SYSTEM 


The RST Branch uses a computerized information system, BASYS, to assist with the selection 
of audits. At the time of our previous audit of the RST Program in 1995, the Ministry was | 
planning to replace BASYS with a new integrated computer system by 1997. In our 1 995 audit, 
we identified some information that BASYS did not produce which would be useful in selecting 
high-risk vendors for audit and recommended that the Ministry ensure that this information be 
considered in the development of its new system. Similarly, the Standing Committee on Public 
Accounts recommended that the new computer system incorporate clearly defined information 
needs, related analysis and implementation deadlines. 


ent audit, the Ministry had not replaced BASYS but was 


However, at the conclusion of our curr | : 
r re-engineering the 


in the process of developing a business case to rationalize the need fo 
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business processes in the Branch, including BASYS. We will be following up on these re- 
engineering initiatives at an appropriate time in the future to see whether the concerns we 
expressed in our 1995 audit, and in this audit, are adequately addressed. 


TAX REFUNDS 


Sales tax that has been paid may be refunded under certain conditions, for example, if the tax 
was paid in error or if a visitor to Ontario has paid tax of $50 or more on goods purchased for 
use or consumption outside the province. During the 1999/2000 fiscal year, the Ministry 
processed approximately 37,500 refund claims having a total value of $130 million. Requests for 
refunds under $100,000 are processed at the Ministry’s head office and requests for refunds 
over $100,000 are processed at regional tax offices. 


We reviewed a sample of refunds issued and found that they were accurately processed and 
properly approved. However, we also noted that, in some cases refunds issued to registered 
vendors were issued “subject to audit” when, for example, eligibility for a refund had not been 
conclusively established. A list of refunds issued subject to audit is to be maintained and 
distributed to regional tax offices for follow-up. 


We found that such a list was neither maintained nor distributed to regional tax offices as 
required. As a result, the Ministry did not know how many refunds were issued subject to audit 
or whether any of them had been followed up and audited. 





Recommendation 


To help ensure that only eligible refunds of retail sales tax are made, the 
Ministry should maintain a list of refunds issued subject to audit and ensure 
that, where refunds are issued subject to audit, the necessary follow-up work 
is performed. 


Ministry Response 





All vendors receiving refunds subject to audit would have been considered 
for audit under the Retail Sales Tax Branch’s normal audit selection process. 
The Ministry will take steps to ensure the listings are prepared ona regular 
basis and distributed to the appropriate offices for consideration in future 
audit selection. 











PENALTIES 


The Retail Sales Tax Act provides for the imposition of a 25% penalty when the correct 
amount of tax is not remitted due to neglect, carelessness, willful default or fraud on the part of 
the taxpayer. Ministry policy requires that this penalty be imposed where the amount of tax 
assessed exceeds a set amount, except in cases where not imposing the penalty is supported by 
clearly documented reasons. 


For the 1998/99 fiscal year, the total amount of penalties imposed was $6 million. We reviewed 


a sample of audit files and found that, for our sample, penalties had not been imposed for 40% 
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of the instances where the assessed tax exceeded the set amount. The reasons for failing to 
impose the required penalties were not documented. 


These penalties are expected to deter vendors from failing to remit the amount of RST owing. 
However, if penalties are not imposed as required, their value as a deterrent will be impaired. 














Recommendation 


To provide an effective deterrent, the Ministry should consider imposing the 
legislated 25% penalty in cases where correct retail sales tax amounts are 
not remitted as a result of taxpayer neglect, carelessness, willful default or 
fraud. When a penalty is not imposed, the Ministry should ensure that the 
reasons are clearly documented. — 


In addition, the Ministry should review and determine whether the threshold 
amount below which penalties are not assessed is appropriate. 


Ministry Response 


We agree that auditors should impose the 25% penalty where applicable or 
document their rationale for not doing so in accordance with the Retail Sales 
Tax Branch’s existing policy. All offices have been instructed to adhere to the 
policy, and audits will be reviewed by quality assurance staff on a periodic 
basis for adherence to the policy. 





The Ministry agrees to undertake a review of the current penalty threshold. 


al 





REVENUE AND RETURNS PROCESSING 


During fiscal 1998/99, approximately 3 million returns were submitted to the Ministry’s Taxation 
Data Centre (TDC) in Oshawa for processing. The Ministry’s Audit Services Branch audits 
TDC operations annually. In their most recent audit report, the Branch concluded that the 
Ministry’s revenue processing was accurate, management practices were effective and the 
system of financial controls for revenue processing continued to ensure timely and accurate 
revenue recording. 

We conducted a walk-through of the returns processing system at TDC and noted that 


adequate controls were in place to ensure returns were reviewed for accuracy and were 
correctly processed. In addition, there was no evidence of a backlog in returns processing. 
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MINISTRY OF HEALTH AND LONG-TERM CARE 


Community Health Centre 
Program 





BACKGROUND 


The Ministry of Health and Long-Term Care’s Community and Health Promotion Branch is 
responsible for administering and funding the Community Health Centre Program. 


Community health centres (CHCs) are non-profit organizations governed by volunteer boards 
of directors. CHCs are funded under the authority of the Ministry of Health Act and provide 
primary health care, health promotion and other health, educational and social services to 
identified priority groups within a given geographical area. These services are tailored to the 
needs of groups within the community that can benefit from enhanced access to health care, 
such as recent immigrants, the elderly and those with low incomes. Unlike most primary health 
care providers, which are funded on a fee-for-service basis, CHCs have fixed budgets and 
provide services using salaried staff. 


For the 1999/2000 fiscal year, the Ministry provided approximately $87 million to fund 56 
CHS, 


AUDIT OBJECTIVES AND SCOPE 


The objectives of our audit of the Community Health Centre Program were to assess whether 
the Ministry had adequate procedures in place: 


* — to ensure that the Program was managed with due regard for economy and efficiency; and 
* to measure and report on the effectiveness of the Program. 


Our audit was performed in accordance with the standards for assurance engagements, 
encompassing value for money and compliance, established by the Canadian Institute of 
Chartered Accountants and accordingly included such tests and other procedures as we 
considered necessary in the circumstances. Prior to the commencement of our audit, we 
identified the audit criteria that would be used to address our audit objectives. These were 
reviewed and agreed to by senior ministry management. 


In conducting our audit, we reviewed and analyzed program policies and procedures, 
interviewed ministry staff and outside experts in the primary health care field, including 
representatives of the Association of Ontario Health Centres, and reviewed relevant literature. 
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The work performed by the Ministry’s Internal Audit Service did not affect our audit because 
the Service had not issued any relevant reports on the Program since 1996. Our audit covered 
the period up to May 2000. 


OVERALL AUDIT CONCLUSIONS 


In our last audit of the Community Health Centre (CHC) Program, conducted in 1994, we 
expressed concern that procedures were not in place to measure and report on the 
effectiveness of the Program. In our current audit, we found that the management information 
system needed for evaluating effectiveness and for financial reporting had been developed but 
was not yet operational. Our other major concerns were: 


¢ The efficiency, effectiveness and ability of CHCs to provide quality care was not being 
assessed by the Ministry or compared with the results achieved by other primary health 
care delivery models. 


¢ Agreements with CHCs did not include measurable objectives, targets and expectations for 
the services to be provided by CHCs. 


¢ Base funding provided to each CHC was not determined by the level of services provided. 


The Ministry did not adjust payments to CHCs to minimize surplus funds held by CHCs at 
year-end. 


¢ Procedures needed to be developed for conducting periodic reviews of CHCs and ensuring 
that CHCs regularly review the quality of care they provide and the services they deliver. 


* Access to information issues related to the management information system needed to be 
resolved. 


DETAILED AUDIT OBSERVATIONS 


CHCs provide health care and health promotion programs to individuals, families and . 

communities (clients). In addition to physicians, a CHC’s multi-disciplinary team may include 

nurse practitioners, community health care workers, psychologists, nutritionists and others. 

CHCs are funded to: 

* promote equity of access to health services; 

* — strengthen the role and responsibility of the individual and the community in health and 
health care delivery; 

* develop coordinated primary health care services that make the most appropriate and 
efficient use of health care providers and health care resources, 

* promote health and prevent illness to enhance the health status of the individuals and 


communities served; and 
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¢ — foster strong links with organizations and groups in the community and work collaboratively 
with them. 


CHCS AND PRIMARY CARE REFORM 


In its May 1996 report to the Minister and the President of the Ontario Medical Association 
(OMA), the Task Force on the Funding and Delivery of Medical Care in Ontario, with ministry 
and OMA representation, recommended that three types of primary care practice models be 
recognized: fee-for-service, capitation and health centre. In addition, the Task Force 
recommended that the effectiveness and efficiency of primary medical care under each of the 
three models should be systematically evaluated and compared. 


While the Ministry has incorporated many of the recommendations of the Task Force, at the 
time of our audit, it had not addressed the role of CHCs in the reform of primary care. 


In April 2000, the Association of Ontario Health Centres recommended that the Ministry invest 
$85 million over the next three years to increase the number of health centres and their 
satellites by 65 and a further $30 million to support existing health centres. In our 1994 audit 
report, the Ministry agreed with our recommendation that it obtain information about the 
effectiveness of the CHC Program before any significant expansion occurred. We continue to 
have this concern. The program evaluation had still not been completed because the information 
needed for the evaluation was not yet readily accessible. 





Recommendation 


To ensure that any additional investments in community health centres 
(CHCs) are justified, the Ministry should first evaluate the efficiency and 
effectiveness of CHCs in providing quality primary health care and compare 
the results with other models of primary care delivery. 





Ministry Response 


During the last decade, many authoritative reports have concluded that the 
integrated, multi-service approach and the emphasis on illness prevention 
used by CHCs are particularly well suited to meeting the needs of 
disadvantaged populations, as well as of people with multiple chronic 
conditions or requiring complex case management. 





The Ministry will ensure that future expansion of the CHC Program is 
appropriately aligned with primary care reform and other strategic health 
directions relating to mental health, rural and northern health, and the 
integration of community health and other human services; and with the 
development of best practices for health promotion and illness prevention. 
The Ministry is undertaking a strategic review of the CHC Program to ensure 
appropriate policy and program alignment with other key strategies. This 
review will identify service delivery approaches used in CHCs, and in similar 
programs in other jurisdictions, where clear evidence of effectiveness and/or 
efficiency has been documented. The Ministry will continue to evaluate the 
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_ experiences of other jurisdictions to identify best practices and approaches 
_ that warrant consideration in Ontario. 












The CHC Program Evaluation System was completed in October 1998. CHCs 

have been collecting information based on evaluation system standards since 

| April 1998 (individual service events) and April 1999 (group and community 
work). The Ministry recognizes that difficulties with data extraction have, to 
date, limited the ability of the system to meet its intended purposes and will 
resolve the data extraction issues by December 2000. 








FINANCIAL MANAGEMENT 


CHCs are required to submit annual budgets to the Ministry that describe the strategic focus of 
their programs and services for the future year and detail their funding requirements. Each 
CHC is also required to submit quarterly financial reports that compare year-to-date 
expenditures to budgeted amounts and activity reports that provide information on the client 
population served and the number and types of client visits. 


No later than three months after the fiscal year-end, CHCs are also required to submit to the 
Ministry annual audited financial statements, including a ministry questionnaire to be completed 
by each CHC’s auditors. As well, they must provide a copy of any management letters issued 
by their auditors and a copy of their annual reports. 


As part of the auditors’ questionnaire, the auditors must confirm that they have reported in 
writing to the CHC’s board any internal control weaknesses that might expose the CHC to a 
material loss of funds or other assets. The auditors must also indicate whether the CHC has 
complied with their previous recommendations. 


We reviewed a sample of CHC files maintained by the Ministry for the 1998/99 fiscal year and 
found that payments to CHCs were properly authorized and paid in accordance with approved 
budgets. However, we also noted that: 


¢ A majority of the audited financial statements were received more than six months after 
the fiscal year-end. 


¢ The majority of CHCs had not submitted the auditors’ questionnaire, and there were no 
management letters in the files we sampled. In the absence of auditors’ questionnaires, it 
could not be determined whether any letters ought to have been on file. 


* While the files contained all required quarterly financial reports, the majority did not contain 
all of the required activity reports. Half did not contain the activity report for the final 
quarter, which includes the activity totals for the year. This limits the Ministry’s ability to 
monitor and analyze CHC activities. 


We also found that, while the Ministry received quarterly and annual financial reports, many 
CHCs had accumulated significant surplus funds. In 1999, consultants were hired by the 
Ministry to determine the amounts owed to it by CHCs. They reported that, as at March 31, 
1998, CHCs owed the Ministry approximately $10.8 million, of which $7.2 million related to 
periods prior to the 1997/98 fiscal year. 
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In the 1998/99 fiscal year, the Ministry recovered $4.2 million in surplus funds. However, as at 
March 31, 1999, CHCs still owed the Ministry $12.6 million. During our review of a sample of 
CHC financial statements for the year ending March 31, 1999, we found one instance where 
the Ministry had incorrectly flowed $456,000 to a CHC but had not included this amount in the 
funds to be recovered. We were advised that this amount would be recovered. By March 31, 
2000, the Ministry had recovered most of the surplus funds that were outstanding from previous 
years. 








Recommendation 


To better enable it to assess whether community health centres (CHCs) use 
their funding economically and in accordance with funding arrangements, the 
Ministry should: . 


* ensure the timely receipt of audited financial statements and activity 
reports; and 


* monitor CHC expenditures during the year and adjust cash flows where 
warranted. 


Ministry Response 


The Ministry agrees with the need to ensure timely receipt of audited financial 
statements and activity reports. Many CHCs do not hold their Annual General 
Meetings until six months into the fiscal year. In those cases, the Ministry 
receives draft statements. To ensure appropriate accountability, it is the 
Ministry’s standard practice to approve each CHC’s annual funding 
agreement only upon receipt and acceptance of audited financial statements. 


Once data transmission from CHCs becomes routine, the Ministry’s 
management information system will produce a detailed series of standard 
service activity reports on a monthly basis. 


The Ministry agrees to monitor CHC expenditures during the year and adjust 
cash flows where warranted. 











ASSESSING CHC PERFORMANCE 


Management Board of Cabinet’s Directive on Transfer Payment Accountability requires an 
effective accountability framework for the prudent management of provincial transfer 
payments. A key principle is that transfer payments should be managed wisely and prudently to 
achieve value for money. “Value” in this context refers to results expected for funds approved. 


The Directive contains a list of mandatory requirements, including defining expectations for 
results to be achieved, which, as stated in the Directive, should “focus more on measurable 
results than on process.” 


The Ministry, after it reviews and approves a CHC’s operating budget, signs a funding 
agreement with the CHC that indicates the amount of the grant and its terms and conditions. 
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We reviewed a sample of CHC operating budgets and funding agreements for the 1998/99 
fiscal year and found that they did not include measurable objectives, the expected amount of 
services to be provided, the number of clients to be served or the anticipated outcomes. We had 
raised similar concerns in our 1994 audit report. 


At the time of our current audit, the Ministry was in the process of revising its standard funding 
agreement with CHCs. The Ministry needs to ensure that its revised funding agreement meets 
the requirements of Management Board of Cabinet’s Directive on Transfer Payment 
Accountability, including measurable objectives and results to be achieved. As well, CHCs 
should continue to be required to adhere to the policies in the CHC Program Resource 
Manual, which sets out accountability requirements for CHCs to the communities they serve 
and to the Ministry. 


The Ministry expects that a new CHC information system will be operational in late 2000. 
While this system will improve the Ministry’s ability to monitor CHCs, the Ministry had not yet 
developed performance indicators and benchmarks for CHCs. Information about costs, client 
characteristics, services provided and outcomes is important for assessing the efficiency, 
effectiveness and quality of care being delivered by CHCs and for assessing the success of the 
programs and services at individual CHCs. At the time of our audit, the Ministry had initiated a 
consultative process with the Association of Ontario Health Centres and the 56 CHCs to 
generate a set of measurable performance indicators. 





Recommendation 


To help ensure the prudent use of funds by community health centres 
(CHCs), the Ministry should: 


e develop measures and benchmarks to monitor and evaluate CHC 
performance; and 

* ensure funding agreements include measurable objectives and the 
_ results to be achieved by CHCs for the funding provided. 


Ministry Response 


The Ministry agrees with the need to meet Management Board requirements 
for transfer payment accountability and is revising its funding agreement 
accordingly. At the time of this audit, the Ministry had initiated a consultative 
process with the Association of Ontario Health Centres and the 56 CHCs to 
generate a set of measures and performance indicators that will be 
incorporated into CHC funding agreements in 2001/02. The data requirements 
of the new management information system (MIS) include furnishing the 
information needed to support this task. The MIS includes service volumes by 
provider type and priority population, as well as data on referrals, 
coordination of care, client characteristics and community partnerships. 
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SERVICES AND STAFFING 


Comparisons of the types and amounts of services provided and their related costs would 
provide information about the relative service efficiency of individual CHCs. Although CHCs 
annually submit budgets to the Ministry detailing the needs in their communities, we found the 
Ministry had not compared budgeted amounts to actual services provided or the number of 
clients served. 


Comparing combinations of health care staff used to deliver programs with standards for 
staffing combinations would also enable the Ministry to ensure that staffing at CHCs is 
adequate and economical in the circumstances. 


In 1999, to assess the relative efficiency of CHCs, the Ministry compared the ratio of CHC 
budgeted annual operating costs to approved staffing levels. While this information was used to 
allocate additional funding to some CHCs, it was not used to determine whether any 
adjustments should be made to the base funding provided to CHCs. 


Most of the funding received by CHCs is used to provide primary health care services. In 
addition to employing primary health care providers, many CHCs employ a variety of health 
care professionals, and as such, develop their own staffing plans. We found that the Ministry 
had not developed standards or guidelines to help CHCs determine cost-effective combinations 
of health care providers. Such information would also assist the Ministry in reviewing the 
appropriateness of current combinations of health care providers at CHCs. 











— 


Recommendation 













To help ensure that community health centres (CHCs) deliver efficient and 
economical health care, the Ministry should: 


¢ obtain and utilize information about the services provided to determine 
the funding level required to provide them; and 


¢ develop guidelines to assist CHCs in determining cost-effective 
combinations of health care staff. 


Ministry Response 


The Ministry agrees that, with the information available in its management 
information system, it will be able to make relative comparisons among 
CHCs and review historic trends in service volumes to assist in determining 
funding levels. 


The Ministry notes that, at this time, CHCs are required to include a variety of 
data and supporting documentation as part of their budget submission, 
including the number of active clients registered, volume of services 
delivered by type of health care provider, major health issues and priority 
populations addressed, major achievements, strategic plans and 
organizational goals. 
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The Ministry will continue to draw lessons from the experiences of other 
jurisdictions in identifying appropriate staffing combinations. For example 
the CHC Program currently funds 87 nurse practitioner positions and will 
participate in evaluating the use of nurse practitioners in delivering primary 
care services in a multi-disciplinary, team-based setting. The Ministry will 
continue to consult as necessary with provider groups and other 
stakeholders to explore combinations of health care providers supported by 
evidence-based research that may contribute to high-quality and cost- 

| effective health care. 
















The Ministry notes that the staffing mix at CHCs must be driven by each 
CHC’s programs and the health needs of the population being served. 








SERVICES FOR NON-INSURED CLIENTS 


Eligibility under the Ontario Health Insurance Plan (OHIP) is not required to access services 
from a CHC. In our 1994 report, we acknowledged that this enables individuals who may not 
have the documentation or may not wish to establish their eligibility under OHIP to receive 
health care. This includes refugees and residents of Ontario without a permanent address. 


Following our 1994 audit, the Ministry instituted a reporting system to monitor use of the funding 
provided to CHCs for specialist services to non-insured refugees. However, at the time of our 
current audit, the Ministry did not have complete information about the number of non-insured 
clients served and the types and number of services that they received from CHCs. This lack 
of information makes it difficult for the Ministry to track the volume and types of services 
provided to non-insured clients and to assess their impact on the CHC Program. 





| Recommendation 


To enable it to better assess the needs of the Community Health Centre 
(CHC) Program, the Ministry should obtain complete information on the 
number of non-insured clients served by CHCs and the types of services 
they receive. _ 


Ministry Response 


The service data collected in the new CHC Program management information 
system will be used to provide the Ministry with a detailed picture of the 
services received in each CHC by non-insured clients, as well as the referrals 


to external providers. 











MONITORING SERVICE DELIVERY 


Currently, CHCs must adhere to the policies contained in the CHC Program Resource 
Manual, which includes a number of requirements for reviewing the provision of services at 
CHCs. However, the Ministry did not conduct periodic formal reviews at CHCs to ensure that 
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the expectations established in the annual funding agreements and the requirements in CHC 
Program Resource Manual were being met. 


Ministry staff informed us that while they had visited CHCs for various reasons, the results 
were generally not documented. The Ministry needs to develop formal procedures for reviews 
of CHCs. We noted that in another jurisdiction, CHCs were subject to periodic external reviews 
as a condition of funding. These reviews included identifying strengths and weaknesses in 
governance, clinical and fiscal operations, and management information systems. 


The CHC Program Resource Manual requires CHCs to develop and regularly review the 
quality of care they provide and the services they deliver, including an assessment of client 
satisfaction. The Ministry did not know whether these reviews were being performed. 


In 1997, the Ministry began to provide funding to the Association of Ontario Health Centres to 
develop an independent accreditation process for CHCs. This process is intended to provide 
assurance that CHCs meet certain quality standards of operation. However, the process is 
voluntary and the results are the property of the CHC. Each CHC can decide whether or not to 
make the results available to the Ministry or the public. 











Recommendation 



















To help ensure that the services provided by community health centres 
(CHCs) are of high quality and are provided cost-effectively, the Ministry 
should: 


° conduct regular reviews of CHCs to ensure that expectations are being 
met; and 


¢ ensure that CHCs regularly review the quality of care they provide and 
the services they deliver. 


Ministry Response 


The Ministry agrees that regular reviews of CHCs would assist the Ministry in 
ensuring that its expectations are being met. Since ministry reviews require 
significant resources, the Ministry will, wherever possible, take advantage of 
systems of review and audit that are already in place. 


CHCs participate in an independent accreditation and quality improvement 
process using the Building Healthier Organizations resource manual 
developed by the Association of Ontario Health Centres. The processes and 
expectations established through Building Healthier Organizations were 
designed in consultation with the Ministry. These reflect transfer payment 
accountabilities and the policies and procedures contained in the CHC 
Program Resource Manual. They also identify a series of standards and 
indicators for gauging and continuously improving organizational health in 
areas such as management and administrative systems, governance and 

| eet Olea processes, programs and services, and community capacity. 
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Although the accreditation process is voluntary, it should be noted that, to 
date, 95% of CHCs have signed participation agreements. Starting with the 
2001/02 fiscal year, the Ministry will require CHCs to report on their 
accreditation status as part of their annual budget submissions. The Ministry 
will also reinforce the requirement that CHCs review the quality of care they 
provide to their clients and include this requirement in the new funding 
agreement. 













COMPLAINTS 


Complaints can provide important information regarding access to services, the quality of 
services provided and the administration of ministry funds. They may also alert the Ministry to 
serious problems at a CHC. 


Between 1994 and 1999, the Ministry hired consultants to conduct organizational reviews of the 
operations of three CHCs. The Ministry had received complaints about the operations at these 
CHCs. The consultants made recommendations about how CHC boards could improve CHC 
operations. For example, in the most recent review, the consultants noted that while protocols 
were in place to guide clinical practice, it was not clear whether a system was in place to 
monitor the extent to which practices were followed. We could find no evidence that ministry 
staff followed up to determine whether action had been taken to address the consultants’ 
recommendations. 


While the Ministry’s usual practice is to refer complaints it receives to the CHC concerned, it 
has not established policies for CHCs to follow when dealing with complaints. Accordingly, 
while individual CHCs may have procedures to address complaints, the Ministry has no 
assurance that these procedures are adequate or consistent. 















Recommendation 


To ensure that complaints concerning community health centres (CHCs) are 
dealt with appropriately, the Ministry should require CHCs to have adequate 
procedures for addressing complaints. 


Ministry Response 


As part of the new funding agreement, the Ministry will require CHCs to have 
complaint procedures. — 
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INFORMATION SYSTEM 
DEVELOPMENT AND IMPLEMENTATION 


In 1994, the Ministry, in collaboration with the Association of Ontario Health Centres, began 
developing an evaluation system for the CHC Program. In 1997, the Ministry began developing 
an information system to collect and analyze data needed for the evaluation of CHCs. 


Special Report: Accountability and Value for Money 


a T 


The development of this information system was divided into two components: the Ministry’s 
management information system (MIS), which was developed by the Ministry’s Information 
Systems Branch, and an information system for CHCs to capture and transfer the required 
information to the MIS. 


In 1996, the Ministry provided funds to CHCs to upgrade their software and hardware. In 1997, 
CHCs were funded to acquire the latest versions of the three software packages then in use. In 
August 1998, staff from the Ministry’s Systems Development Branch and their consultant 
discovered significant problems with the hardware, software and data integrity at several CHCs 
they visited. 


In the fall of 1998, the vendor of the clinical management software used by the majority of 
CHCs became insolvent and its assets, including future product development initiatives, were 
sold. The new owner indicated that within a year it would no longer support the software 
product installed at CHCs. As a result, the Ministry implemented a plan to design and install at 
each CHC local databases that are in line with the data structures of the Ministry’s MIS. As 
well, the Association of Ontario Health Centres initiated a process to replace operational 
system software being used at CHCs with a system to be supplied by a single vendor. The 
Association hired a consultant to assist it with the selection process. 


Based on the information we received from the Ministry, there was a lack of coordination and 
collaboration among those responsible for implementing this new operational system software 
and converting information from existing CHC systems. While the Ministry’s Systems 
Development Branch was consulted on some technical matters, it had no direct involvement in 
the process for selecting a vendor. Although data from predecessor systems has been loaded 
onto the Ministry’s MIS, difficulties in extracting data from the new system has delayed its 
implementation. At the time of our audit, the system was not expected to be fully operational 
until late 2000. 

















; 
Recommendation 


To help ensure that information systems are properly developed and 
implemented, the Ministry should ensure that appropriate oversight and 
project management expertise is applied. 





Ministry Response 


The Ministry’s Systems Development Branch has been regularly consulted on 
technical matters and regularly informed about the process to replace the 
operational system software. The Ministry agrees that the Systems 
Development Branch could have been more fully involved in decisions 
pertaining to the replacement of the operational systems in community health 
centres (CHCs). The Ministry will ensure that the technical knowledge and 
project management skills of the Systems Development Branch are more 
fully utilized in the future. 








a —  — ee 


158 Office of the Provincial Auditor 




















3.08 


eee 
ACCESS TO INFORMATION 


While the information system is expected to be in operation in late 2000, the Ministry has not 
yet addressed essential matters relating to it, such as access to the information and the 
ownership and use of the data. We were informed by ministry staff that its Legal Services 
Branch had been requested in March 2000 to draft an agreement to be signed by the Ministry, 
the Association of Ontario Health Centres and CHCs concerning the sharing of data related to 
programs and services delivered to clients. 


While the Ministry’s management information system (MIS) captures information about the 
primary health care services that CHCs have provided to their clients, this information will not 
include individual health insurance numbers or other identifying information, pending the 
introduction of proposed legislation on the confidentiality of personal health information. The 
lack of such information will make it difficult to integrate and analyze the use of health care 
services by individuals. The Ministry also requires such identifying information to manage and 
evaluate the effectiveness of the CHC Program and for health care planning. 


Ministry staff informed us that, where available, CHCs collect and record a patient’s health 
insurance number and that an appropriate data field exists in CHC databases and the Ministry’s 
management information system to do so. However, no formal requirement exists for this 
information to be provided to the Ministry. 





Recommendation | 


The Ministry should expedite the resolution of any access to information 
issues to ensure that their impact on the new information system is 
recognized and addressed early in the development process. 


Ministry Response — 
The Ministry agrees with this recommendation and is working with legal 
counsel to resolve this issue. 


Any future decision to collect health numbers will be tied to ministry 
corporate data standards and the passage of legislation related to the 
collection, access, use and disclosure of health records. 
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MINISTRY OF HEALTH AND LONG-TERM CARE 


Emergency Health 
Services 





BACKGROUND 


The provision of ambulance services in Ontario is governed by the Ambulance Act. Under the 
Act, the duties and powers of the Minister of Health and Long-Term Care include ensuring “the 
existence throughout Ontario of a balanced and integrated system of ambulance services and 
communication services used in dispatching ambulances.” Central ambulance communication 
centres (CACCs) dispatch all land ambulances. Base hospitals train, certify and provide on-the- 
job medical direction to paramedics. Only ambulance services certified under the Ambulance 
Act may operate in the province. 


On January 1, 1998, under Local Services Realignment, the province transferred the 
responsibility for funding land ambulances to municipalities. The responsibility for operating 
local land ambulance services was to be transferred from the province to municipalities by 
January 1, 2000. On March 23, 1999 the province announced it would now fund 50% of the 
approved costs of land ambulances, retroactive to January 1, 1999. The deadline for 
municipalities to assume responsibility for providing land ambulance services was extended to 
January 1, 2001. The Ambulance Act states that every municipality will “be responsible for 
ensuring the proper provision of land ambulance services in the municipality in accordance with 
the needs of persons in the municipality.” At the time of our audit, five municipalities had 
already assumed responsibility for operating their ambulance systems. 


The province will continue to pay the full cost of dispatching ambulances, land ambulance 
services for the First Nations and air ambulance services. 


The Ministry will remain responsible for ensuring that minimum standards are met for all 
aspects of ambulance services. The provincial share of funding of ambulance services is 
provided through the Ministry’s Population Health and Community Services Program. During 
the 1999/2000 fiscal year, Emergency Health Services’ expenditures, prior to recoveries by the 
province from municipalities for their portion of the operating costs, were approximately 

$404 million. 
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AUDIT OBJECTIVES AND SCOPE 


The objectives of our audit of Emergency Health Services were to assess whether: 


* adequate procedures were in place to ensure compliance with legislation, policies and 
procedures and to measure and report on the effectiveness of ambulance services, 
including whether the realigned ambulance system will be balanced and integrated as 
required by the Ambulance Act; and 


* resources were acquired and managed with due regard for economy and efficiency. 


Our audit was performed in accordance with the standards for assurance engagements, 
encompassing value for money and compliance, established by the Canadian Institute of 
Chartered Accountants, and accordingly included such tests and other procedures as we 
considered necessary in the circumstances. Prior to the commencement of the audit, we 
identified the audit criteria we would use to address our audit objectives. These were reviewed 
and accepted by ministry senior management. 


Our audit focused on the activities at the Branch’s head office as well as regional offices, 
CACCs and a sample of base hospitals. Our audit was substantially completed in March 2000. 
We reviewed and, where warranted, relied on work completed by the Ministry’s Internal Audit 
Service. 


OVERALL AUDIT CONCLUSIONS 


The Ministry faces significant challenges, including potential service impairments, in ensuring 
that a municipally operated land ambulance system is balanced and integrated. These 
challenges include meeting the province’s Local Services Realignment goal of improved 
accountability and better service at a lower cost to Ontario taxpayers, at a time when the 
provincially operated system was already not meeting response time requirements. We also 
noted that municipal representatives were concerned that the province should not be 
downloading services that were not meeting response time requirements. Our major concerns 
were as follows: 


¢ In 1998, the last full year for which statistics were available, over 50% of land ambulance 
operators did not meet response times that were required by regulations under the Act, 
even though these requirements were based on their actual performance in 1996. The 1996 
ambulance response times varied widely. The Ministry estimated that an additional 
$40 million annually and $11.6 million in one-time funding would be needed to meet the 


current response time requirements. 

* The Emergency Services Working Group reported that, for the period it surveyed, hospitals 
requested redirect consideration or critical care bypass even though 36% of the time their 
emergency departments were not at full capacity. 

In addition, during 1999, one region experienced approximately 1,900 instances where 
patients in serious but stable condition or in life or limb threatening condition waited from 
to 45 minutes in the ambulance outside hospitals before the hospitals accepted them. 
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Incidents like those cited above increase the risk of poor ambulance response times for 
subsequent patients. 


* Municipal boundaries may impair the seamlessness of the delivery of ambulance services. 


* The Ministry estimated that, due primarily to the transfer of land ambulance services to 
municipalities, the cost of providing the existing level of service would increase by 
approximately $53 million in the year 2000. One municipality that assumed responsibility for 
its land ambulance services stated that providing the same level of service will cost 
approximately $2 million more than the provincially operated system. 


* One-time costs of realignment include an estimated $25 million in compensation to land 
ambulance operators and $39 million in other costs. 


¢ The Ministry had not defined which municipal land ambulance costs would qualify as 
approved costs for provincial funding. 


In addition, to improve the management of resources, the Ministry needed to ensure that: 


* ambulance service providers, central ambulance communication centres and base hospitals 
are meeting ministry standards; 


* it works with municipalities to implement standards for transporting patients using the most 
appropriate and cost-effective method; and 


¢ where applicable, air ambulance patients are equitably billed on a timely basis, and amounts 
owing to the Ministry are collected. 


DETAILED AUDIT OBSERVATIONS 


LAND AMBULANCE 
LOCAL SERVICES REALIGNMENT 


The Province of Ontario’s 1998/99 Annual Report stated that “the goal of Local Services 
Realignment is to improve accountability, reduce waste and duplication, and provide better 
government services at a lower cost to Ontario taxpayers. Provincial and municipal services 
are being realigned in order to provide the best possible services at the lowest possible cost.” 


Several studies and reports have raised concerns that the realigned land ambulance system will 
not provide a balanced and integrated system of services, as required under the Ambulance 
Act, and will be more costly to Ontario taxpayers. For example, in September 1996, the 
Ministry raised concerns that differences in the quality of care and service may appear 
between municipalities and across the province due to differences in tax bases, organization and 
sophistication. In November 1996, the Sub-Panel on Emergency Services of the Who Does 
What Panel recommended that “the province should continue to fund and control ambulance 
services as part of the health care system. This would maintain a seamless system in the area 
of health care with a consistent level of care and service province-wide.” The Panel had been 
appointed by the then Minister of Municipal Affairs and Housing to make recommendations on 


162 Office of the Provincial Auditor 


3.09 


renters 


how best to overhaul the delivery and funding of many provincial and municipal government 
services in Ontario. 


In August 1998, the Land Ambulance Transition Task Force reported that increased municipal 
control over dispatch could benefit patients and taxpayers. In contrast, in 1998, the Provincial 
Base Hospital Advisory Group stated that downloading land ambulance services to 
municipalities “will raise borders that do not currently exist. These borders may lead to gaps in 
service and a mosaic of service provision throughout our province.” In January 1999, 
consultants reported to the Ministry that devolving dispatch to a large number of independent 
municipalities rather than maintaining the current 19 CACCs “would likely result in a highly 
fragmented system with boundary service problems, little opportunity for efficiency gains and 
the potential for a negative impact on overall patient care throughout the province.” 


The consultants also stated that municipalities will likely attempt to gain cost efficiencies that 
may not be in the best interests of ambulance services province-wide. Similarly, in December 
1999, the Ontario Hospital Association noted that separate segments tend to look after their 
own needs without considering the future needs of the whole ambulance system. 


TRANSITION 


In March 1997 the Ministry formed the Land Ambulance Transition Task Force to provide 
advice on the transfer of ambulance services to municipalities. The Task Force adopted, and 
the Ministry agreed to, five fundamental principles for the delivery of land ambulance services: 


¢ Seamlessness: the closest available, appropriate ambulance should respond to a patient, at 
any time, in any jurisdiction, regardless of any boundaries. 


¢ Accessibility: municipalities should ensure reasonable access to ambulance services and 
that ambulance services respond regardless of the location of the request. 


* Accountability: ambulance services should be medically, operationally and financially 
accountable to the municipalities and the Ministry. 


* Integration: emergency and transfer services should be integrated with other health care 
Services. 


* Responsiveness: ambulance services should be responsive to fluctuating health care, 
demographic, socioeconomic and medical demands. 


In August 1998, the Task Force made recommendations for ensuring the continuation and 
enhancement of a patient-focused ambulance system. Key recommendations dealt with 
implementing a quality-assurance-based certification system for operators, reviewing 
ambulance dispatch and creating a committee to provide ongoing support and advice on 
maintaining a seamless ambulance system. Most of the Task Force’s recommendations were 
accepted by the Ministry and, at the time of our audit, were in varying stages of 
implementation. 


In early 1999, the Ministry and the Association of Municipalities of Ontario established the 
Land Ambulance Implementation Steering Committee (LAISC) whose terms of reference 
included identifying the information and implementing the tools needed to facilitate, monitor and 
evaluate the transfer. The LAISC is also responsible for making recommendations to the 
Minister concerning legislation, policies and procedures relating to municipal responsibility for 


land ambulance services. 
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At the time of our audit, agreements with municipalities that had already assumed responsibility 
for their ambulance services did not include funding arrangements. The Ministry also needed to 
develop performance measures and reporting requirements to ensure the existence of an 
integrated ambulance system that provides consistent accessibility and services to all Ontario 
residents. 





Recommendation 


The Ministry should ensure that after realignment has been completed, the 
land ambulance program in Ontario is seamless, accessible, accountable, 
integrated and responsive. 


The Ministry should also take corrective action where necessary. 


Ministry Response 


Upon assumption of responsibility for land ambulance services, 
municipalities sign a Memorandum of Agreement with the Ministry. The 
Ambulance Act and Regulations clearly establish standards for ambulance 
operations. 





The Ministry will monitor through operational reviews, inspections, 
investigations and central ambulance communication centres the transfer of 
responsibility. 


In July 2001, the Ministry, through field offices of Emergency Health Services, 
will assess the impact of realignment. 





Where necessary, corrective action will be taken. 











RESPONSIBILITY FOR DISPATCH 


The Ministry is responsible for the full cost of operating the 19 central ambulance 
communication centres (CACCs) in Ontario, which coordinate and direct the movement of all 
land ambulances. During the 1998/1999 fiscal year, CACC expenditures totalled approximately 
$29 million. 


According to the Ministry, ambulance service in Ontario has historically been a seamless 
system that crosses all municipal boundaries without reference to residence or other 
demographic factors. 


As part of the realignment, municipalities providing ambulance services outside their own 
boundaries can bill other municipalities for these services. Unless the municipalities agree on a 
rate, billings are based on each municipality’s average cost per call. These costs can vary 
significantly. At the time of our audit, the Ministry was calculating and processing all such 
billings for municipalities. 


A Dispatch Sub-Committee of the Land Ambulance Implementation Steering Committee was 
established to consider options for governance, management and control of land ambulance 
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dispatch services, including the possible transfer to municipal management. In December 1999, 
the Sub-Committee agreed on a number of principles, including: 


¢ The number one priority of dispatch is to ensure that ambulances are available to handle 
emergency calls in each jurisdiction and within the provincially mandated response times, 
recognizing that the dispatch system drives effectiveness. 


¢ Municipalities should have the right to manage dispatch, but should not be forced to. 


* Municipalities should control dispatch protocols, procedures or policies that will apply to 
their ambulance fleets, but the principles of an accessible, integrated, seamless, accountable 
and responsive system must be upheld. 


However, the Ontario Hospital Association, also in December 1999, maintained that ambulance 
dispatch services should remain a provincial responsibility to ensure that both emergency and 
non-emergency ambulance services are coordinated and seamless to patients. 

















/ Recommendation 


The Ministry and municipalities should work to ensure that municipal 
boundaries do not impair the delivery of ambulance services to patients or 
_add significantly to costs. 


Ministry Response 
The Ministry, through field offices of Emergency Health Services, is now 
_working with municipalities to address boundary issues. In addition, central 
ambulance communications centres have committees, which include 
municipal representation, that discuss the need for seamless ambulance 
service. As municipalities assume responsibility for land ambulance services, 
the Ministry agrees to work closely with them to manage this issue. 





COSTS OF REALIGNMENT 


A September 1998 consultants’ report confirmed the Ministry’s estimate that downloading land 
ambulance services to municipalities would increase the total annual cost of providing these 
services in the province by approximately $12 million in 1999 and an additional $53 million in 
2000. This did not include estimates of the annual costs incurred by municipalities to hire staff 
to administer their ambulance services or increased wage rates resulting from competition for 


paramedics. 


In January 1999, consultants reported to the Ministry that inter-municipal billing for services 
would add additional cost and complexity to the system with no added value in services. 


The December 1999 minutes of the Land Ambulance Implementation Steering Committee 
(LAISC) indicated that one municipality, having conducted a request for proposals for its 
ambulance service, found that the least expensive option for maintaining the existing level of 


service will cost almost $2 million more than what the province paid to operate the service. 


Se ee 


165 
Special Report: Accountability and Value for Money 








In addition, transferring the responsibility for land ambulance services to municipalities has 
resulted in one-time transition costs. 


Since municipalities will be able to choose whether to continue with their current ambulance 
operators, an independent panel was established to determine the level of compensation the 
Ministry should pay to private operators for the loss of their business relationship with the 
Ministry. A consulting firm reported to the Minister that, in the past, private sales of ambulance 
operations were between 2.5 and 3.5 times the annual amount of the operator’s management 
compensation plan. The management compensation plan was the amount each operator 
received for management and administrative services and was based on each operator’s call 
volume. 


The panel awarded each operator 5.5 times the management compensation plan amount based 
on their call volume in the 1998/99 fiscal year. This resulted in 67 operators being paid a total of 
approximately $25 million, or an average of about $370,000 per operator. 


Other one-time costs include an estimated $15 million to be paid by the Ministry on behalf of 
ambulance operators as penalties for cancelling or settling leases and severance costs of 
$24 million for their employees. 


FUNDING 


Although the province announced in March 1999 that it will fund 50% of the approved costs of 
land ambulance services effective January 1, 1999, at the time of our audit, approved costs had 
not yet been defined. Until these costs are defined, the ability of the Ministry and municipalities 
to budget for future program costs is limited. However, before it determines what should 
comprise approved costs, the Ministry needs to compare and analyze the current costs for each 
ambulance service in relation to the level of service provided. 


In November 1999, a LAISC Costing Sub-Committee was established to outline a process to be 
followed by the Ministry in determining approved costs. The Sub-Committee’s terms of 
reference provided that the approach adopted exclude provincial micro-management or line-by- 
line budget reviews. However, the Ministry still needs to ensure that funding is reasonable 
when compared to the service being delivered. 


In December 1999, the municipal members of the Sub-Committee proposed that a global budget 
funding model, using the previous year’s budget (based on actual costs for the municipalities to 
deliver services), be used as the base budget. They also proposed additional funding based on 
the quality and quantity of the services as determined by an assessment of needs. Accordingly, 
only additional funding would be based on an assessment of needs on a municipality-by- 
municipality basis. 


At the time of our audit, the Sub-Committee had not yet outlined a process for the Ministry to 


determined approved costs, and the Ministry and municipalities had not finalized the budgeting 
and budget review processes. 
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»Recommendation 













To help ensure that funding provided to municipalities is reasonable and 
equitable, the Ministry should: 


* develop a process that assesses relative need and ensures equitable 
funding across the province; and 


¢ define which municipal costs will qualify for provincial funding. 
| Ministry Response 


The Ministry with the Land Ambulance Implementation Steering Committee 
has established a sub-committee to review and make recommendations on 
standards and costs. 


| The Ministry has now defined the basis for determining which municipal 
costs will qualify for provincial funding. 








RESPONSE TIMES 


Calls for ambulances are generally prioritized at the CACCs as follows: 




















Urgent call 
Prompt call 

Scheduled call 
Deferrable call 


Life or limb threatening 
Serious but stable or under professional care 
Inter-institutional transfer | 

Delays not detrimental to patient safety 














Source: Ministry of Health and Long-Term Care 


Fast responses with properly trained and equipped personnel are critical to the survival or well 
being of patients with certain types of illnesses or injuries. A swift response is especially critical 
for cardiac arrest victims. The Ministry’s database of patient problems indicates that at least 
11% of code 4 calls in 1998 related to heart attack patients. 


In 1994, the Ministry initiated the Ontario Pre-Hospital Advanced Life Support (OPALS) study 
to assess the impact on survival rates of cardiac patients receiving defibrillation within 

8 minutes of call receipt and other advanced life support interventions within 11 minutes of call 
receipt. The study is being conducted from 1994 to 2002 in 20 communities in Ontario. Interim 
results indicated that rapid defibrillation has led to a 33% increase in patient survival (from 


3.9% to 5.2%). 


The U.S. National Institute of Health recommends that a first responder should arrive at the 
scene less than 5 minutes from the time of dispatch 90% of the time and that advanced life 
support should arrive within 9 minutes. According to the American Heart Association, very few 
resuscitation attempts are successful as little as 10 minutes after a heart attack. 


The OPALS study also plans to evaluate survival compared to the optimal time for 
defibrillation. That is, whether the optimal standard from call receipt to defibrillation should be 


ee 


167 
Special Report: Accountability and Value for Money 





8 minutes for 90% of the patients, or whether the time frame should be shortened to 7 or fewer 
minutes in order to increase survival rates. 


AMBULANCE RESPONSE TIMES 


At the time of our audit, regulations under the Ambulance Act prescribed land ambulance 
response times (from call receipt by land ambulance operator to arrival at the scene) for code 4 
calls based on actual response times achieved in 1996. The regulations required that each land 
ambulance operator “‘shall ensure that, in 90 per cent of the priority 4 (emergency) calls 
received in a 12-month period, the response time performance of the operator’s ambulance 
service is equal to the response time performance set by the person who operated the service 
in 1996.” Response times for other types of calls were not covered by regulations under the 
Act. 


We found that there were wide ranges in code 4 response time requirements throughout the 
province. As well, there were inconsistent requirements within municipalities where there was 
more than one operator. 


In December 1999, the LAISC stated that 1996 response times might not be the ideal standard. 
Its Costing Sub-Committee also noted that service and response times in similar jurisdictions 
were uneven across the province. For example, one municipality was concerned that its 1996 
response times were 50% longer than those of a similar-sized jurisdiction. 


According to the Ministry’s data, in 1998 over 50% of the operators had not met their required 
response times. Data from the first half of 1999 indicated that over 60% of the operators were 
not meeting their required response times. Municipal representatives on the Costing Sub- 
Committee expressed concerns that the province should not be downloading services that were 
not meeting code 4 response time requirements when the province was operating the system. 


The Ministry estimated that it would cost approximately $40 million in additional annual 
operating costs and $11.6 million in one-time costs for all services to meet the 1996 response 
time requirements. 


Effective May 1, 2000, new regulations made under the Ambulance Act no longer specify 
response time requirements for ambulance operators. However, under the new regulations, 
operators must meet the criteria set out in the Ministry’s Land Ambulance Certification 
Standards, which currently require operators to meet requirements based on 1996 response 
times. 





Recommendation 


To help ensure that ambulance response times for emergencies meet the 
needs of patients throughout the province, the Ministry, together with the 
municipalities, should: 





« review current response time requirements for reasonableness and 
consistency and, where necessary, make adjustments; and 


e take appropriate corrective action where specified response time 
requirements are not met. 
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Ministry Response 











The Ministry and municipalities will jointly review standards, including 
response times, and make recommendations if changes are in order. 


To assist municipalities in determining whether response time requirements 
| are being met, the Ministry has conducted a comprehensive review of 

response times and is now providing municipalities with access to statistical 
data. 








DISPATCH RESPONSE TIMES 


At the time of our audit, call taker and dispatcher response times for code 4 calls were 
established in regulations under the Ambulance Act. The call taker was required to obtain the 
necessary patient information to accurately prioritize the call and assign it to a dispatcher within 
45 seconds. The dispatcher must accurately select and alert the land ambulance crew within 
one minute and 15 seconds. There were no legislated requirements for other types of calls and 
there were varying time limits in responding to them. 


We noted that in 1998, which was the most recent information available at the time of our audit, 
none of the 17 CACCs where the Ministry tracked the time from call receipt to assignment to a 
dispatcher met the 45 seconds timeframe. Fifteen of the CACCs had exceeded the 45 seconds 
timeframe by more than 25%. Of the 18 CACCs where the Ministry tracked the time for the 
dispatcher to alert the ambulance crew, 11 did not meet the one minute and 15 seconds 
requirement. Five of the CACCs exceeded the one minute and 15 seconds requirement by 
more than 25%. Although there was no overall requirement, of the 17 CACCs for which times 
were available, only three contacted the ambulance operator within two minutes of receiving 
the call, while 11 exceeded the requirement by more than 25%. 


Although the Ministry was aware that the dispatch response time requirements were generally 
not being met, we saw minimal evidence of corrective action being taken. 


Effective May 1, 2000, new regulations made under the Ambulance Act no longer specify 
response time requirements for dispatch. 













Recommendation 


To better meet the needs of patients, the Ministry should: 


* establish dispatch response time standards; 
* monitor whether these standards are being met; and 
¢ take timely corrective action where necessary. 


Ministry Response 


In recognition of red tape reduction, dispatch standards were removed from 
the Ambulance Act and will be incorporated into a performance agreement to 
be entered into by each dispatch centre and the Ministry of Health. 
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Adherence to the performance agreement standards will be closely 
monitored and corrective action will be taken. Considerable extra resources, 
including training and technical staff, will be implemented in each dispatch 
centre to assist with standard compliance. In addition a technical group Is 
working on a review of the Priority Card Index and a new computer-aided 
dispatch system will be implemented over the next four years. 


Subject to agreement, corrective action will be taken, as necessary. 














REDIRECT CONSIDERATION AND CRITICAL CARE BYPASS 


In municipalities with more than one hospital emergency department, a hospital may: 


¢ instruct CACCs to redirect non-critical patients to an alternate hospital (redirect 
consideration); or 


* notify the CACC that it is unsafe for the hospital to accept any new ambulance patients as 
patient care will be compromised (critical care bypass). 


Reaching another hospital because of redirect consideration or critical care bypass usually 
increases travel time and may delay treatment. We were informed that the Ministry had not 
analyzed the impact of redirect consideration and critical care bypass on travel time or the 
delays in reaching the next patient. 


In April 1998, the Emergency Services Working Group, with representation from the Ontario 
Hospital Association and the Ministry, reported that hospitals were requesting redirect 
consideration and critical care bypass at different occupancy levels and for different reasons 
and that there were no standard, monitored criteria. For example, the Working Group reported 
that, during the period it surveyed, hospital emergency departments were not at full capacity 
36% of the time when redirect consideration or critical care bypass was declared. 


In February 1999, the Ministry issued Standards for Ontario Hospital Emergency Units 
Providing Ambulance Access. The standards state that “ambulance diversion should not result 
in ambulance travel time beyond 15 minutes and in ambulance travel distance beyond 20-25 
km.” If all hospitals within a given area are on critical care bypass, patients should be taken to 
the closest hospital. 


One region experienced approximately 1,900 instances during 1999 where code 3 and 4 patients 
waited from 15 to 45 minutes in an ambulance outside the hospital until the patient was 
accepted by the hospital. In some cases the delay was up to two hours. 





Recommendation 


The Ministry should analyze the impact of redirect consideration and critical 
care bypass on ambulance services, including response times for 
subsequent patients, and, where necessary, take appropriate corrective 
action. 
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Ministry Response 













The Ministry addressed the impact upon the Toronto Ambulance Service 

| through a ten-point plan implemented in 1999. Components of the plan are 
| being considered for implementation in other areas of the province where 
redirect consideration and critical care bypass are extensively utilized. 


| The Ministry has now established a new working group to review redirect 
| consideration and critical care bypass, including its impact on ambulance 
‘Services. . : 








DISPATCH PRIORITY 


An effective dispatch protocol enables call takers and dispatchers to rapidly identify patient 
problems, assign priority codes, select an ambulance to respond and provide instructions to 
callers. When a call is received at a CACC, the call taker uses a dispatch protocol to obtain 
critical information needed to assess patient priority. 


To accomplish this, approximately 15 years ago the Ministry developed the Dispatch Priority 
Card Index. Since then, base hospitals and CACCs have updated the Index. As a result, 
different versions were in use at the time of our audit, and one CACC was using a 
commercially available dispatch protocol. 


When using the Index, the call taker documents the nature of the patient problem in a database. 
However, we noted that in the 1998 database, 30% of the code 4 calls and 49% of the code 3 
calls listed patient problems as “unknown.” We noted that, while calls were recorded, there was 
no overall regular monitoring of call-taker compliance with the Index’s protocols. 


We were informed that the Index’s questions often do not trigger the correct priority response. 
While under-prioritizing may jeopardize patient safety, over-prioritizing places stress on the 
ambulance system and may affect overall response times. 


In February 1999, a base hospital advised the Ministry that, from a 10-month review, it found 
that call takers using the Index did not identify cardiac arrest in 20% of known cardiac cases. 
The base hospital stated that the Index “has not been validated in the same way that other 
commercially available dispatch and call-taking algorithms have been.” In May 1999, the 
Provincial Base Hospital Advisory Group recommended that the Ministry commit resources to 
replace the Index. 


The Ministry has established a working group whose terms of reference, drafted in early 2000, 
included recommending changes to improve the effectiveness, efficiency and outcome 
accuracy of the Index. In March and April 2000, workshops were held to update dispatch 
questions. We were informed that changes resulting from these workshops would be 
implemented over the next few years in conjunction with a new dispatch computer system. The 
Ministry plans to implement the system starting in the fall of 2000, with all dispatch centres 


converted by 2003. 
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Recommendation 


The Ministry should ensure that central ambulance communication centres 
appropriately assess and prioritize patient needs. 


Ministry Response 


The addition of 10 dispatch training coordinators across the province will 
assist with refinement of prioritization of calls. In addition, as was mentioned 
earlier, a technical group is working on a review of the Priority Card Index and 
a new computer-aided dispatch system will be implemented over the next 
four years. 





PERFORMANCE MONITORING 


The evaluation of ambulance services involves the examination of many clinical conditions and 
outcomes, including death, disability and discomfort. With respect to performance monitoring, it 
would be reasonable to expect that: 


¢ the cost effectiveness of ambulance services is evaluated: 


* new treatments and services are implemented only after their effects have been 
demonstrated; and 


* performance is compared with operators in other parts of the province of similar size, 
geography and demographics. 


Information on operator performance and the impact of ambulance services will become even 
more important as responsibility for land ambulances is transferred to the municipalities. The 
Ministry will need this information to help ensure the effectiveness of Ontario’s ambulance 
system. With the exception of communities participating in the OPALS study, which primarily 
focuses on performance targets for cardiac patients, the Ministry does not obtain and analyze 
data on the impact of ambulance services on patient outcomes. To facilitate data collection, a 
system is needed that integrates information from emergency services and other health care 
providers. 








Recommendation 
To help ensure that the land ambulance system effectively meets patient 
needs, the Ministry should: 


* research systems to analyze operator performance, including its impact 
on patient outcomes; and 


¢ take corrective action where necessary. 
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Ministry Response 





The Ministry has now developed a certification process that focuses primarily 
on patient care provided by operators. The impact on patient outcomes 

requires extensive scientific studies such as the Ontario Pre-hospital 
Advanced Life Support study. 


The certification process for ambulance operators under the Ambulance Act 
provides for action where a contravention of standards has occurred. 
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SERVICE REVIEWS 


Ambulance Operator Service Reviews 


In 1993, the Ministry initiated a process for reviewing operational and administrative activities 
of ambulance operators. The goal was for all ambulance operators in Ontario to be reviewed 
within five years. The first cycle of the service reviews was completed by the end of 1999. 


Operators are notified 90 days before the review is conducted, and are sent an information 
package on the scope of the review. We noted that in one Canadian province, 25% of 
certification inspections are performed without prior notice. 


Ministry policies for reviews require follow-up visits to ensure that corrective action was taken 
where necessary; however, the policies do not specify when the follow-up should occur. We 
noted that the Ministry did not track the timing or results of follow-ups and that few were 
conducted prior to 1999. Of the 61 operators that were followed up in 1999: 


¢ approximately 60% were for reviews that had occurred more than three years before; and 
¢ 26% of these operators required further follow-up. 


Approximately 60% of the follow-ups that we reviewed did not ensure that all previously 
identified problems were corrected; for example, some recommendations were recorded as 
being not applicable or were deleted without explanation. 


Service reviews often directed the operator to arrange for follow-up through the Ministry’s 
regional office. The majority of ministry regional managers we surveyed indicated that their 
responsibilities included ensuring that recommendations were addressed. We were informed 
this was generally done through informal discussions. 


Ministry policy states that if noted deficiencies cannot be satisfactorily resolved, the Ministry’s 
investigations unit may be called and punitive action may follow. However, there were no 
criteria identifying when an operator should be referred to the investigations unit or when an 


operator’s licence should be revoked. 


Effective May 1, 2000, regulations under the Ambulance Act require that all ambulance 
operators be certified at least once every three years. The certification process will have many 
of the same requirements as, and replaces, the service review and licensing process. 
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Recommendation 


To help ensure that ambulance operators meet ministry requirements, the 
Ministry should: 


* consider performing certification reviews without advance notice to 
increase assurance of consistent quality of practice by operators; 

e have a coordinated follow-up of all deficiencies identified during 
certification reviews on a timely basis; and 

° clarify the circumstances when a formal investigation of an operator is 
required and when a certificate should be revoked. 








Ministry Response 


The concept of certification was based upon allowing adequate notice for 
operations to prepare for the review. However, the Ministry is 
institutionalizing an inspection process based upon random inspections 
without notice. 


Deficiencies will be followed up by Emergency Health Services field offices 
and the Ministry Inspections Group. 


The Ministry reviews every complaint received by the Ministry with the intent 
of ensuring that ministry and legislative requirements are being met. Where 
there is substantial evidence that requirements are not being met, a formal 
investigation will be conducted. If a complaint falls under some other 
jurisdiction, it will be referred to the proper authority. Certificates will be 
revoked where a contravention of the Ambulance Act standards has occurred. 








Central Ambulance Communication Centre and 
Base Hospital Reviews 


We were informed that, as of December 1999, all CACCs and base hospitals were to be 
reviewed once every three years. While no policy previously existed for the frequency of 
CACC reviews, base hospitals previously required reviews every five years. 


We noted that 37% of the CACCs had been reviewed between 1996 and 1998. In all cases 
improvements were necessary. Two of these CACCs were found to be seriously deficient in 
meeting dispatch requirements, such as rarely using the Dispatch Priority Card Index system. 
This could result in incorrect patient and dispatching prioritization. Only one of these two 
reviews was followed up to ensure the recommendations were met. The follow-up did not 
occur until one year later. 


Only two of the 21 base hospitals had been reviewed: one in 1995 and one in 1999. While the 
1999 review raised no major issues, the 1995 review concluded that there was a failure of 
effective communication among the base hospital, emergency medical attendants and the 
service operators. There was no formal follow-up of this review. 


ee eeeeeeeeSSSSFeFesesesesee 


174 Office of the Provincial Auditor 


see 
Effective January 1, 2001, regulations under the Ambulance Act require communication 
(dispatch) services not operated by the Ministry and base hospitals to enter into an agreement 


with the Ministry for the provision of services. We were advised that the ministry-run CACCs 
will be subject to similar performance requirements. 





Recommendation 


To help ensure that emergency patient needs are being effectively and 
consistently addressed, the Ministry should: 


* review central ambulance communication centres and base hospitals 
within reasonable timeframes; and 


e resolve all identified deficiencies on a timely basis. 
‘Ministry Response 


The Ministry will develop schedules to ensure that operational reviews are 
| conducted within reasonable timeframes. In addition, continual review of 
_central ambulance communication centres will take place through field 

_ offices and inspections. : 








Identified deficiencies are discussed with the operators/managers and 
corrective action plans are developed. Corrective action taken will be 
_monitored. — 





| COMPLAINTS 


Complaints received by the Ministry’s investigations unit are generally logged and assigned to 
an investigator. We were informed that approximately 100 investigations are conducted 
annually. We found that investigations were generally completed on a timely basis. Once 
completed, investigation reports are usually forwarded to senior management and the 
appropriate regional office. 


Our review found that, while files indicated that investigations were generally thorough, certain 
key information was not consistently tracked in the complaint log. This included whether an 
investigation was undertaken and the outcome. 


Follow-ups enable the Ministry to ensure that noted deficiencies are corrected and | 
requirements are being met. Ministry policy does not adequately describe who is responsible for 
following up deficiencies noted in investigation reports, the extent of regional office involvement 
or the timing of the follow-ups. In addition, there is no requirement to send follow-up results to 


the investigations unit. 
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Recommendation 





To better enable it to assess whether complaints are satisfactorily resolved, 
the Ministry should: 


¢ establish clear lines of responsibility for following up on deficiencies 
identified in investigation reports; and 


° ensure that follow-ups are completed and documented. 





Ministry Response 


A process for investigating complaints received by, or referred to, another 
jurisdiction relating to land ambulances has now been developed for 
presentation to the Land Ambulance Implementation Steering Committee. 


Follow-ups will be completed and documented. 








NON-EMERGENCY INTER-INSTITUTIONAL TRANSFERS 


Most non-emergency calls are for transfers of patients between health care facilities such as 
hospitals, nursing homes and homes for the aged. Transfers are only performed when 
ambulances are not required for emergency calls. 


In 1995 the Ministry of Transportation issued a report on the potential for more effective use of 
ambulances. The report noted that, due to emergency calls, ambulances were not able to meet 
the demand for non-emergency transportation. 


In March 1997, the Ministry of Health issued a transportation guide to hospitals noting that 
ambulances should be used for non-emergencies if a medical practitioner decides a patient is 
medically unstable, requires a medical escort and needs a stretcher. Otherwise, less costly 
alternatives such as taxis, stretcher-capable private medical transport services and volunteer 
agencies should be considered. 


In December 1999, the Ontario Hospital Association reported an increased use of unregulated 
patient transportation services in larger urban areas. Although draft guidelines on alternative 
medical transportation services were prepared for municipalities jointly by the ministries of 
Transportation and Health and Long-Term Care, we were advised that there were no plans to 
issue these guidelines because, under the Highway Traffic Act, non-ambulance medical 
transportation 1s a municipal responsibility. 








Recommendation 


The Ministry and municipalities should jointly develop and ensure standards 
are in place that address passenger safety and encourage the use of the 
most cost-effective resources for transferring non-emergency patients. 
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Ministry Response 





Land ambulance non-emergency and inter-institutional transfers are the 
responsibility of municipalities to administer and ensure cost-effectiveness. 
Emergency Health Services field offices will work with municipalities to assist 
in their decisions in this area. Existing vehicle standards for ambulances were 
designed to address patient and crew safety and will be used to provide 
guidance. 





AIR AMBULANCE PROGRAM 


The air ambulance program was established in 1977 to transport patients who were 
inaccessible by land ambulance or where transport by land ambulance was too time-consuming. 
Air ambulances are also used to transfer medical teams and organs for transplants. The 
Ministry contracts with private operators of helicopters and airplanes to provide air ambulance 
services through three different arrangements: 


¢ Dedicated contracts require the air carrier to provide the aircraft and pilots, and the 
Ministry provides the paramedics and air bases, including the aircraft hangar and crew 
facilities. 


¢ Preferred provider contracts require the air carrier to provide all services, including 
paramedics and air bases. 


¢ Standing offer agreements require carriers to provide all services; however, these carriers 
may decline a request for an air ambulance and are used to supplement the dedicated and 
preferred services. 


For the 1998/99 fiscal year, ministry expenditures for the air ambulance program totalled 
approximately $37 million. 


USE AND SELECTION OF AIRCRAFT 


The Ministry has established a process for assessing the need for an air ambulance. Once the 
need is determined, the dispatcher prioritizes available aircraft based on flight time and 
estimated cost, and then determines which aircraft to use based on patient need. 


Air dispatch policies and procedures require that, where the recommended aircraft is not 
available, the next most appropriate aircraft be used and the selection process documented. The 
prioritized list of possible aircraft is to be printed and retained. 


While the Ministry has developed criteria for selecting an air ambulance over a land ambulance, 
we found that the reason was generally not documented and documentation was lacking to 
support the aircraft selected. In addition, the prioritized list of possible aircraft, although 
required, was not printed. Without this information, the Ministry cannot ensure an air ambulance 
was used appropriately and that the aircraft selected met the patient’s needs in the most 


economical manner. 
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Recommendation 











The Ministry needs to better demonstrate through proper documentation 
that air ambulances are used appropriately and that the aircraft selected 
meet patient needs in the most economical manner. 


Ministry Response 


The Emergency Health Services Branch has developed a new system, 
including a new software application, that will capture and provide the 
necessary documentation for the selection of appropriate air ambulance 
services best suited to the patients’ needs in the most economical manner 
available. 





RESPONSE TIMES 


Air ambulances are primarily dispatched through a central air dispatch centre. At the dispatch 
centre, the call taker determines whether the call is an emergency and transfers the call to the 
appropriate dispatcher. 


We noted that there were no response time standards for either the call taker to transfer the 
call to the dispatcher or for the dispatcher to contact the air carrier. Although a computer 
system is used to capture patient and flight information, the time when the call taker received a 
call was generally not tracked. In addition, we found that the time the dispatcher contacted the 
air carrier to arrange a flight was recorded only 26% of the time. Since these times are not 
tracked, the Ministry cannot monitor the reasonableness of the air dispatch centre’s response 
times. 


Response time requirements were included in each air carrier’s contract. All contracts allowed 
for delays due to extenuating circumstances such as poor weather conditions. There were no 
penalties if response times were not met. 


The following table shows the response times for each type of contract and how often those 
response times were met for code 4 calls. 
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Response Requirement and Compliance for Code 4 Calls, 


by Contract Type, 1999 
Contract Type Response Requirement staples 
e 
Dedicated 


En route within 10 minutes of me 
accepting flight cae 
En route within 10 minutes of 
accepting flight 

Accept/decline within 10 minutes of 
Ministry’s request 

En route within 30 minutes of 
Ministry’s request 
































Preferred 68% * 










Standing Offer Agreement 









46% ** 






<60% ** 


* 


Ministry data 
** Office of the Provincial Auditor estimate based on available data 


As expected, we found that response times for code 3 calls were even longer. 





Recommendation 
To help ensure that air ambulance dispatch and response times meet the 
needs of patients, the Ministry should: 
¢ develop, track and monitor air ambulance dispatch response time 
standards; 
¢ track and monitor contracted air carrier response times and take 
_corrective action when necessary; and 
* ensure air carrier contracts contain appropriate penalties for not meeting 
_ required response times. 


| Ministry Response 


The Emergency Health Services Branch has now implemented an incident 
reporting requirement with all carriers when response times exceed contract 
requirements. 


The Branch will more closely monitor compliance by air carriers with 
response time requirements. 


The Branch will meet with carriers to consider possible changes to 
agreements related to response times. 
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INSPECTING SERVICE PROVIDERS 


The air ambulance policies and procedures manual requires that all air ambulance providers be 
inspected annually. Areas inspected include aircraft maintenance, crew training, 
communications systems, safety, medical equipment and airbase facilities. 


We found that annual inspections were not being completed. Dedicated airplanes and 
helicopters under multi-year contracts were last inspected in 1994 and 1995 respectively. For 
standing offer agreements covering the period October 1, 1998 to September 30, 1999, only 9% 
of the airbases in our sample were inspected. Where inspections had been completed, there 
was no evidence of follow-up to ensure that noted deficiencies, such as the cleanliness of 
medical equipment and the aircraft, had been corrected. 


Information on air carrier use and performance is required in order to evaluate the 
appropriateness of the cost and services supplied. The Ministry received monthly and annual 
reports from dedicated air ambulance bases; however, for air carriers that provide services 
under the preferred provider contract or standing offer agreements, reports were not received 
or produced by the Ministry on their use and performance. 


We reviewed the two-year contract extensions for the dedicated helicopter and airplane 
services, whose initial five-year terms expired in March and September 1999 respectively. The 
contract extensions were contingent on annual program evaluations. These evaluations were 
required to forecast program activity as well as review the previous year’s service quality, 
contract cost, aircraft type, maintenance and appropriateness of base locations. However, the 
Ministry renewed both contracts without conducting program evaluations. 








Recommendation 





To help ensure that the air ambulance program is providing safe and quality 
services at an appropriate cost, the Ministry should: 


¢ conduct inspections and evaluations of air ambulance providers in 
accordance with ministry policies and procedures; 

e track and analyze air ambulance use and performance data; and 

¢ take corrective action when necessary. 








Ministry Response 


As with land ambulance operators, the air ambulance providers will be 
required to re-certify every three years through a standard quality review 
process. The Ministry is in the process of increasing the number of 
inspectors and investigators to address aviation-related matters. 








The previously mentioned new computer-aided dispatch system and the 
additional resources that are being engaged in the Medical Air Transport 
Centre will allow for analysis of air ambulance use and performance data. 


Corrective action will be taken subject to the terms of the relevant agreement 
with a carrier and be subject to the certification standards under the 
Ambulance Act. 
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PATIENT BILLINGS 


Individuals are invoiced for the cost of their air ambulance trip if they are not covered under 
OHIP or when the trip is not medically necessary. After an air ambulance trip has been 
completed, the hospital determines if the flight is billable by the Ministry and forwards 
applicable documentation. 


Standing offer agreement carriers forward information to the Ministry on all of their flights. 
However, hospitals are the sole source of information for flights taken on dedicated and 
preferred provider aircraft. 


We noted that the Ministry did not have a process in place to ensure hospitals forwarded 
information on all billable patients in a timely manner and therefore could not ensure all patients 
who should have been billed were invoiced. At the time of our audit, no billing documents had 
been received from hospitals for preferred provider flights, which commenced operations in 
September 1999. 


The Ministry is responsible for determining flight costs and issuing invoices. Patients travelling 
on standing offer agreement aircraft were invoiced for the actual cost of the trip, which 
averaged $3,400. However, billings for flights on dedicated aircraft were based only on the time 
the patient was on board the aircraft, and averaged $650. The Ministry had not established a 
billing practice for preferred provider flights. 


In the 1998/99 fiscal year, 98 invoices were issued totalling approximately $273,000, of which 
approximately $140,000 was collected. Of the invoices we sampled, 56% were issued more 
than two months after the flight. 


There was no process in place for following up on outstanding accounts. Generally, there was 
no correspondence with the patient after the initial invoice was sent. Uncollected accounts 
were not forwarded to the province’s Collection Management Unit, which pursues overdue 
accounts. 





Recommendation 


To help ensure that, where applicable, all patients are billed equitably and 
outstanding amounts are collected, the Ministry should establish effective 


procedures to: 

° ensure that all patients who should be billed are identified and invoiced 
in a timely manner for the total cost of the service provided, regardless of 
the air carrier used; and 

¢ collect outstanding accounts on a timely basis. 


Ministry Response 


The patient billing criteria and system is currently under review with the 
intention to develop a more equitable billing system based on actual costs 


incurred. 
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The Ministry is currently setting up a new aging system to help monitor the 
accounts receivables more closely and incorporate the services of the 


province’s Collection Management Unit as necessary to ensure effective 
collection of outstanding patient billings. 
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MINISTRY OF HEALTH AND LONG-TERM CARE 


Health Service 
Organization and Primary 
Care Network Programs 





BACKGROUND 


The Ministry of Health and Long-Term Care’s Alternative Payment Programs Branch is 
responsible for the administration and funding of the Health Service Organization (HSO) 
Program and the Primary Care Network (PCN) Program. Both programs are funded under the 
authority of the Health Insurance Act. 


Primary health care is usually defined as the care provided at the first point of contact between 
a patient and the health care system. This care can be provided by family physicians, nurse 
practitioners and others. 


The HSO Program was established in 1973. Each HSO comprises physicians who have agreed 
to provide a defined set of primary health care services to patients enrolled as members of that 
HSO. As of March 2000, 58 physician-sponsored and 5 non-profit HSOs were providing 
primary health care services to approximately 310,000 enrolled individuals. 


In 1999, the Ministry introduced PCNs as a new model for providing primary health care 
services. The May 2000 Ontario Budget stated that the government’s goal is “to have 80% of 
eligible family doctors practising in PCNs over the next four years.” 


Each PCN comprises a group of physicians who provide defined primary health care services 
to enrolled patients. At the time of our audit, pilot PCNs were being tested at seven sites 
throughout Ontario. The PCN Program is intended to provide easier access to primary health 
care services and better coordination of health care information as well as to help reduce waste 


and duplication in the health care system. 


During the 1999/2000 fiscal year, the Ministry, through its Health Insurance Program, provided 
transfer payments totalling approximately $75 million to HSOs and $11 million to PCNs. 


AUDIT OBJECTIVES AND SCOPE 


The objectives of our audit were to assess whether: 


¢ the Health Service Organization Program was managed with due regard for economy, 
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* — satisfactory procedures were in place to measure and report on the effectiveness of the 
Health Service Organization Program; and 


* the Ministry had adequate procedures in place to ensure that the Primary Care Network 
Program was being established with due regard for economy, efficiency and effectiveness. 


Our audit was performed in accordance with the standards for assurance engagements, 
encompassing value for money and compliance, established by the Canadian Institute of 
Chartered Accountants and accordingly included such tests and other procedures as we 
considered necessary in the circumstances. Prior to the commencement of our audit, we 
identified the audit criteria that would be used to address our audit objectives. These were 
reviewed and agreed to by senior ministry management. 


In conducting our audit, we reviewed and analyzed program policies and procedures, 
interviewed ministry staff and outside experts in the field of primary health care, reviewed 
relevant literature and researched the delivery of primary health care in other jurisdictions. We 
also reviewed the work performed by the Ministry’s Internal Audit Service. However, because 
the Service had not issued any recent reports on the programs, we were unable to reduce our 
audit work. Our audit was substantially completed in April 2000. 


OVERALL AUDIT CONCLUSIONS 


The Ministry had not fully implemented the recommendations from our 1994 audit of the Health 
Service Organization (HSO) Program. Accordingly, many of our major concerns for this 
current audit were similar to those we noted in 1994. 


In order to ensure that the HSO Program is managed economically and is effective in meeting 
its objectives, the Ministry needed to: 


* — periodically verify HSO patient rosters to ensure HSOs are not being overpaid. The only 
time that roster verification was undertaken, approximately 8,000 of the 18,000 patients 
selected for verification were determined to be ineligible; 


* ensure that HSOs comply with roster and fee-for-service billing limits and, where 
applicable, recover any overpayments; and 


* complete its study comparing HSO health care utilization and costs with fee-for-service 
practices. 


With respect to the Group Health Association (Association), the Ministry had not conducted a 
formal assessment to determine whether it was obtaining value for money for the more than 
$20 million in annual funding it provided to the Association. 


We concluded that in order to ensure that the new Primary Care Network (PCN) Program is 
established with due regard for economy, efficiency and effectiveness, the Ministry needs to: 


* — ensure that, in line with the planned expansion of the Program, it carefully assesses the 
interim report on the evaluation of the PCN pilot projects and the results of the study 
comparing HSO utilization and costs with fee-for-service practices; 
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° review options for adjusting capitation (per person) funding rates to ensure that fundine 
provided to PCNs fairly reflects the levels and various types of service being provided: 


* — assess the effect that different primary care funding models have on the appropriateness of 
physician referrals of patients to specialists; 


* ensure that the maximum number of patients each physician is allowed to enrol is 
reasonable and that rosters are periodically verified; 


* develop performance measures and standards to monitor and evaluate the cost- 
effectiveness of PCNs;: and 


¢ develop a health performance information system that meets the needs of both the Ministry 
and PCNs. 


DETAILED AUDIT OBSERVATIONS 
HEALTH SERVICE ORGANIZATIONS 


Under the terms of a standard agreement with the Ministry, health service organizations 
(HSOs) agree to provide defined primary health care services to the patients enrolled (rostered) 
with them. Patients rostered with an HSO must be eligible for Ontario health insurance (OHIP) 
and must agree that they will normally use that HSO as their source of primary health care 
services. 


The HSO Program’s stated objectives include: 


¢ developing a coordinated system of health care delivery that is accessible, efficient and 
economical and makes the most appropriate use of health care resources; 


* enhancing the health status of the population through health maintenance and illness 
prevention measures; and 


* creating an environment that is supportive of physicians and other health care personnel. 


HSOs receive monthly payments from the Ministry based on the number of patients on their 
rosters. These payments are calculated using per-person (capitation) rates that are based on 
the age and sex of patients on a roster. These payments do not vary with the volume of 
services provided. In addition to their capitation payments, HSO physicians are paid by OHIP 
on a fee-for-service basis for services provided to non-rostered patients. 


ROSTER VERIFICATIONAND NEGATION 
ROSTER VERIFICATION 


Because payments to HSOs are based on the number of enrolled patients as reported by each 
1@ HSOs are not overpaid. There 1s a risk 


HSO, accurate patient rosters are essential for ensurir erp ‘ 
that patients who no longer use a particular HSO could be kept on the HSO’s roster eee 
the Ministry will continue to pay for those patients as long as they are rostered. In our | 4 )4 

audit, we noted that HSO rosters were not being properly verified. At that time, the Ministry 
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indicated that it would conduct audits of HSO rosters and confirm with patients their continued 
membership in HSOs. 


However, in our current audit, we continued to have concerns about the inadequate verification 
of patient rosters. Patients enrolling or terminating their enrolment with an HSO are required to 
sign and date a form. While the information on the forms is submitted electronically to the 
Ministry, the forms are kept at the HSO. In that regard, the Ministry conducts signature audits 
to verify, on a sample basis, whether HSOs had obtained signed and dated forms for all of their 
rostered patients and whether the information submitted to the Ministry was correct. Between 
1995 and 1998, ministry staff completed signature audits of 23 HSOs. However, no signature 
audits have been completed since August 1998. 


We found that, for the signature audits that had been completed, required procedures were not 
always followed and corrective action was not always taken. For example, in one audit 
approximately 50% of the enrolment forms examined at an HSO were either not signed or not 
dated. The Ministry had not followed up to ensure that the HSO had subsequently obtained 
properly completed forms from its patients. 


In addition, the standard agreement between the Ministry and the HSOs allows the Ministry to 
require an HSO to annually verify up to 5% of its roster to determine whether the patients are 
entitled to remain on an HSO’s roster. Based on the results, the Ministry may request additional 
verification. 


Roster verification has been undertaken only once. The Ministry selected approximately 5% of 
the patients from each HSO’s March 1997 roster for verification by the HSO. The Ministry 
selected patients at high risk of being ineligible to belong to the HSO they were enrolled with, 
for example, patients who may have died or who likely no longer resided in the HSO’s 
geographical area. 


We found that ministry staff had not compiled or analyzed the results of the roster verification 
nor prepared any formal reports for ministry senior management. We summarized the results 
and noted that: 


* 44% or about 8,000 of the 18,000 patients selected for verification had to be removed from 
HSO rosters; and 


* 16 HSOs had to remove 60% or more of the patients that they had verified. Of these, four 
HSOs had to remove more than 80% of their sample. 


Ministry funding to these HSOs was reduced to reflect these roster changes. 


Based on the Ministry’s risk criteria, some HSO rosters had more than 5% of their patients in 
the high-risk group. Yet only the initial 5% were verified. Also, HSOs that had to remove a high 
percentage of their patients from their rosters were not requested to perform additional 
verifications. 


NEGATION 


If a rostered patient obtains health care services outside the HSO that the HSO was expected 
to provide, a deduction (negation) is made from a subsequent payment to the HSO. Currently, 
the amount of the negation is 50% of the actual amount paid by OHIP for the services that 
were obtained outside the HSO by a rostered patient. 
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However, negation does not provide a financial incentive for an HSO to remove a patient from 
its roster unless the amount of negation is greater than the payment the HSO received for that 
patient. In addition, negation does not apply when services that an HSO was funded to provide 
are obtained from physicians who do not bill OHIP, for example, physicians employed by 
community health centres. Also, negation will not be effective if the patient has left Ontario or 
has died but has not been removed from OHIP’s Registered Persons Database. 




















Recommendation | 


To help ensure that payments to health service organizations (HSOs) are 
made only for patients actually receiving their primary care from the HSO, the 
Ministryshould: 

oe implement regular verification procedures for HSO rosters; and 

© where practical, apply negation for primary care services provided to 

___ rostered patients by alternatively funded physicians. 


Ministry Response 


The Ministry plans to implement a comprehensive system of roster 

| verification that will be compatible with the new Primary Care Network (PCN) 
_ system and will enable verification to take place on a more regular basis. The 
Ministry is also analyzing current signature audit policies and procedures for 
_HSOs for the development of the overall verification system process. Both of 
these actions will aid in ensuring that payments to HSOs are made only for 

| eligible patients. — : 


: Where feasible, negations for primary care services provided to HSO rostered 
patients by alternatively funded physicians will be implemented. 


scl 





ROSTERAND BILLING LIMITS 


The HSO Program underwent a period of rapid growth in the late 1980s. However, in 1991, the 
Ministry stopped the Program’s expansion due to concerns about its cost-effectiveness. In 
1993, the Ministry and the Ontario Medical Association agreed to limit HSO rosters to 2,500 
patients per full-time-equivalent HSO physician. While fee-for-service billings for non-rostered 
patients were limited to $50,000 per annum for an individual HSO physician, the overall limit for 
each HSO was capped at $30,000 per full-time-equivalent physician in the HSO group. 


In our 1994 audit of the HSO Program, we recommended that the Ministry establish 
procedures to ensure that HSOs comply with agreed-upon limits. The Ministry responded that 
procedures were under development and would be implemented shortly. 


We compared HSO roster and fee-for-service billing limits to information maintained by the 


Ministry for the 1997/98 and 1998/99 fiscal years and found that several HSOs had exceeded 
their limits. Based on this information, ministry staff subsequently estimated that overpayments 
for the 1997/98 and 1998/99 fiscal years totalled approximately $200,000 for exceeding fee-for- 
service billing limits and $133,000 for exceeding roster limits. 


Es 
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Recommendation 


To ensure that health service organizations (HSOs) are paid in accordance 
with agreed-upon roster sizes and fee-for-service billing limits, the Ministry 
should monitor HSO compliance with those limits. 


Ministry Response 


Fee-for-service billing limit recoveries for the two previous years will 
commence in the fall of 2000. 


Roster limit recoveries have previously been difficult to implement due to the 
retroactive nature of the last HSO agreement and the grandparenting of HSOs 
from the previous contract period. Generally, exceeding roster limits has not 
been a serious problem as HSOs that exceed their limit normally reduce their 
roster quickly through attrition. However, the Ministry will ensure that roster 
limits are complied with in the future. 
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INSTITUTIONAL SUBSTITUTION PROGRAM GRANTS 


In 1993, the Ministry introduced Institutional Substitution Program (ISP) grants for HSOs to 
provide rostered patients with additional health care services not covered by capitation 
payments, such as nutritional and mental health counselling. In the 1999/2000 fiscal year, the 
Ministry provided ISP grants totalling approximately $8 million to fund services provided by 
nurses, dietitians and social workers. 


In 1997, ministry staff evaluated the programs and related services funded through ISP grants 
and concluded that ISP grants were generally meeting their objectives. However, they also 
identified grants totalling $500,000 that should be cancelled or reduced, but these grants 
continued to be paid. 


In 1999, the Ministry commenced another review of ISP grants. The results of this review 
were to be compared to the results from 1997. Recipients that were chronically not meeting 
their grant proposal targets, providing inappropriate services or serving an extremely small 
proportion of rostered patients were to have their grants reduced or cancelled. At the time of 
our audit, this review was still ongoing. 


We also noted that, while the Ministry has permitted HSO physicians who join primary care 
networks (PCNs) to keep their ISP grants, other physicians in PCNs have not been provided 
with any similar compensation. This may create inequities in services and potentially 
complicates the evaluation or comparison of primary care networks. 








Recommendation 


To help ensure that value for money is received from Institutional 
Substitution Program grants, the Ministry should: 


¢ examine the appropriateness of the grants; and 
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¢ review the grants being provided and, where necessary, make 
appropriate adjustments. 










Ministry Response 


The Ministry plans to conduct a formal review of all Institutional Substitution 
Program grants and make appropriate adjustments where warranted. The 

Ministry will allocate resources to ensure that a thorough review takes place. 
_ The 1999 review lacked sufficient data to enable the development of 
appropriate recommendations. 











PERFORMANCE MEASUREMENT AND REPORTING 


In 1991, ministry staff reviewed the HSO Program and recommended that a committee with 
HSO and ministry representation be established to develop measures for assessing the quality 
of HSO services and for examining existing information systems to ensure that they were 
providing effective and useful feedback on performance. The review also indicated that the 
Ministry should ensure that HSOs meet the requirements of an established quality assurance 
program and establish performance contracts with HSOs. 


According to the Ministry’s HSO program manual, HSOs are expected to regularly conduct 
routine medical and professional audits of the quality of care provided by their staff. We could 
find no evidence to indicate that the Ministry had ever determined whether HSOs have 
complied with the requirement regarding quality of care audits. In addition, no performance 
contracts had been established with HSOs. 


In our 1994 audit report, we recommended that the Ministry investigate the feasibility of 
developing a system to measure and report on the effectiveness of the Program. The Ministry 
had commenced a study to compare health care utilization and costs in HSOs with fee-for- 
service practices. At the end of our current audit, this study, which the Ministry had originally 
anticipated would be completed by October 1995, was still unfinished. We understand that 
problems with data have reduced the scope of the study and delayed its completion. 













Recommendation 


The Ministry should complete its study comparing health care utilization and 
costs in health service organizations with fee-for-service practices and 
ensure that the results are considered in the implementation of the Primary 


Care Network Program. 


Ministry Response 


The Ministry has supplied data to the Centre for Health Economics and Policy 
Analysis (CHEPA) that meet the study's requirements for claims history. 
Ministry staff are currently working on rebuilding historical roster and rate 
tables to facilitate a final data transfer to CHEPA so that the study can be 


completed. 


ee 
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GROUP HEALTHASSOCIATION 


Since 1963, the Group Health Association (Association), which is unique in Ontario, has 
operated a community-based, ambulatory care facility that includes diagnostic services in Sault 
Ste. Marie. While the Association’s funding is provided and administered by the HSO Program, 
the Association does not operate as an HSO. 


The Ministry and the Association have negotiated a funding agreement whereby the 
Association receives $18.8 million annually for the first 39,000 patients on its roster and $375 
for each additional patient up to a total of 44,000 patients. In the 1999/2000 fiscal year, the 
Association received $20.7 million, the maximum amount permitted under the agreement. 


In delivering services, the Association contracts with an independent partnership of physicians 
who provide both primary and specialist medical services to its rostered patients. These 
physicians can also bill OHIP for services provided to patients who are not on the Association’s 
roster. 


FINANCIAL AND OPERATIONAL REPORTING 


In 1994, the Ministry engaged a consultant to determine the reasons for the deficits incurred by 
the Association in the previous two years and to advise the Ministry on appropriate funding. 
The consultant’s recommendations included funding the Association on a program basis and 
negotiating a new contract that contained commitments to improve productivity and utilization. 
These included establishing a minimum number of patients per full-time-equivalent family 
physician, collaborating on health system planning and developing outcome measures. The 
consultant also recommended that the Ministry review, at least annually, the Association’s 
workload and its financial performance. 


Currently, the Ministry does not have sufficient information to assess whether it is obtaining 
value for money for the more than $20 million in annual funding it provides to the Association. 
For example, it has not received detailed, annual operational planning or budget information, 
although, apparently, under the terms of its agreement with the Association, it could request 
these. Further, we understand that the Ministry has not reviewed the agreement between the 
Association and the independent partnership of physicians since 1984. Therefore, the Ministry 
has not been in a position to assess the reasonableness of the compensation paid to the 
independent partnership, which received approximately $7.5 million from the Association in the 
1998/99 fiscal year. 


The most recent contract with the Association was for five years, beginning on April 1,1995. 
Under this contract, the Association undertook to establish and achieve productivity and 
utilization improvement targets. During the first year it was to determine benchmarks for 
comparing productivity achievements throughout the remainder of the term of the agreement. 
Although the Association was to have annually reported to the Ministry on the achievement of 
its targets, we found no evidence that it had done so. 


The Ministry did not request the Association’s audited financial statements for the fiscal years 
1993/94 to 1998/99 until November 1999. Moreover, since these annual statements were 
prepared on a consolidated basis, they did not provide detailed information on expenses by 
program. In addition, while the Association is also required to submit quarterly financial reports, 
it had not done so. 
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OUTSIDE MEDICAL SERVICES AND ROSTER VERIFICATION 


Under its agreement with the Association, the Ministry is entitled to monitor the extent to which 
the Association’s rostered patients use outside medical services. However, unlike HSOs, 
payments made to the Association are not reduced when its rostered patients obtain services 
outside the Association that the Association is funded to provide. Consequently, the Association 
has no incentive to remove patients from its roster. 


In 1998, the Association agreed to verify samples of its roster as at September 1, 1998 and 
July 1, 1999. The verifications were to be completed by March 1, 1999 and January 1, 2000, 
respectively. Although the Ministry agreed to provide a list of rostered members to be verified, 
at the time of our audit, it had not done so and the verifications of the Association’s roster had 
not been completed. 


In March 1999, the Health Services Restructuring Commission (HSRC) reported on the role 
and impact of the Association on local health services utilization. The HSRC noted that it had 
expected to find that the Association’s rostered patients made little use of outside medical 
services, particularly the services of family physicians. However, it found that, in the 1995/96 
fiscal year, the Association’s rostered patients used over $6.1 million in physician services 
outside the Association. Of this amount, $4.6 million was for services provided by specialists 
and $1.5 million was for services provided by family physicians. The HSRC advised the 
Ministry that it should review the Association’s programs and services, with particular emphasis 
on the use of medical resources outside of the Association by rostered patients. 


The HSRC also concluded that a joint executive committee with representation from the 
Association and area hospitals should be created. The committee was to focus on joint planning 
to maximize integration and realign services where necessary to minimize duplication. The 
Ministry was advised to make future funding conditional on the Association’s participation in the 
committee. At the time of our audit, we were advised that such a committee was being 
established. 







Recommendation 


To help ensure that it is receiving value for the funding it provides to the 
Group Health Association (Association), the Ministry should: 


° require the Association to provide it with sufficient information so the 
Ministry can assess whether the Association’s programs and services 

were delivered efficiently and effectively; 

* ensure that the accuracy of the Association’s roster is verified 

periodically; and 

* assess the reasonableness of the outside use of medical services by the 

‘Association’s rostered patients and implement a process that enables the 

‘Ministry to reduce payments to the Association where warranted, as Is 

currently done in the Health Service Organization Program. 
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Ministry Response 


The Ministry agrees with the recommendations and will work with the Group 
Health Association to develop an accountability framework to help set 
benchmarks, deliverables, indicators/measurement tools and performance 
targets by March 31, 2001. Preliminary discussions (verbal and written) have 
already taken place with the Association and requested financial and service 
information is being sent to the Ministry. The Ministry will also develop 
mechanisms to review roster accuracy. The Ministry will ensure that 
agreements with the Association enable the Ministry to recover funds where 
warranted. 











PRIMARY CARE NETWORK PROGRAM 


In November 1998, the Ministry received Management Board of Cabinet approval for the 
implementation and funding of five primary care pilot programs called primary care networks 
(PCNs). 


PCNs are established through standard agreements with the Ministry. Participation is voluntary 
for both physicians and patients. A PCN is made up of a group of physicians who join together 
to provide a defined set of primary health care services. PCN physicians may maintain their 
own offices and provide primary health care to their rostered patients. However, those patients 
have access to other physicians in the PCN when their own physician is unavailable. Physicians 
in a PCN are linked by a software network that allows them access to the medical histories of 
rostered patients in the absence of a patient’s primary physician. Patients also have 24-hour 
telephone access to medical advice and to medical care after hours and on holidays and 
weekends. 


To establish a PCN, participating physicians must enrol 40% to 60% of their patients within six 
to twelve months. Rostered patients must reside within a defined geographical area, be eligible 
for OHIP, and agree not to enrol with another physician for six weeks and not to change their 
enrolment more than twice a year. 


PCNs can choose to be compensated for the services their physicians provide either on a 
capitation basis, where physicians receive monthly payments for each of their rostered patients, 
or through reformed fee-for-service, where payments are only made if services are actually 
provided. Reformed fee-for-service uses the number of enrolled patients to establish an annual 
maximum billing limit. 


PCNs also receive funding from the Ministry for enrolment costs and telephone health advisory 
services. They are also entitled to financial assistance to acquire the information technology 
they require to establish their provider network. In its new four-year agreement with the 
Ontario Medical Association, effective April 1, 2000, the Ministry agreed to contribute funding 
for the acquisition of information systems. The May 2000 Ontario Budget announced 

$150 million for new information systems to support the transition to primary care networks. 


In addition, PCN physicians are entitled to fee-for-service payments for services they provide 
to enrolled patients that are not included in the defined set of primary care services to be 
provided by the PCN as well as for services they provide to non-rostered patients. 


192 Office of the Provincial Auditor 











ce ee ee 


Although the PCN Program is still in a pilot stage of development, the government has indicated 
that it is to be rapidly expanded in the near future. Specifically, the May 2000 Ontario Budget 
stated that the government's goal is “to have 80% of eligible family doctors practising in PCNs 
over the next four years.” The importance of the primary health care system, in terms of both 
government expenditures and its significance for the quality of life for Ontarians, requires that 
any changes to its delivery be carefully considered. The Ministry’s goal is to improve the 
delivery of primary health care without incurring higher expenditures for physician services. 
The discussion and recommendations below are intended to assist the Ministry in ensuring that 
the Primary Care Network Program is established with due regard for economy, efficiency and 
effectiveness. 


IMPLEMENTING PRIMARY CARE NETWORKS 


In November 1998, when it approved implementation and funding for the first five pilot PCNs, 
Management Board of Cabinet directed the Ministry to assess the effectiveness of the health 
care delivery provided by the pilot PCNs and report back on the results of that assessment prior 
to expanding the PCN Program. 


In December 1999, the Ministry issued a request for proposal for the evaluation of the pilot 
sites, indicating that the final report would be due by March 30, 2003, with various interim 
reports required at earlier dates. At the time of our audit, the Ministry had selected consultants 
to undertake the evaluation. However, even if the timelines for the evaluation are met, it is 
possible that the planned expansion of the PCN Program will be substantially completed before 
the final results of the evaluation are known. Nevertheless, the interim reports of the pilot 
evaluation may provide some pertinent information, as would research into the experiences of 
other jurisdictions with similar programs. 


An ongoing ministry-funded study comparing health care utilization and costs in HSOs and fee- 
for-service practices may also produce some useful information to guide the expansion of 
PCNs, although this study has been delayed because of problems with data. 





Recommendation 


To help ensure that the Primary Care Network (PCN) Program provides 
improved primary health care in a cost-effective manner, the Ministry should, 
in line with the planned expansion of the Program, carefully assess: 


¢ the interim reports on the evaluation of the PCN pilots; 
¢ relevant experiences of other jurisdictions; and 


° the results of the ministry-funded study comparing health care utilization 
and costs in health service organizations and fee-for-service practices. 


Ministry Response 


Formal evaluation of the primary care reform pilot project will consist of a 
number of interim reports followed by a final report slated for delivery in 


March 2003. The goals of primary care reform that are to be addressed by the 


evaluation include improved primary care access, improved quality and 
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continuity of primary health care and increased patient satisfaction with and 
cost-effectiveness of health care services. 


The Ministry is committed to continually reviewing the experiences of other 
jurisdictions to identify best practices that warrant consideration in Ontario. 
The Ministry is currently represented on the Federal/Provincial/Territorial 
Advisory Committee on Health Human Resources, which has reviewed 
relevant international literature, and will participate in national and 
international conferences to draw lessons on primary care reform from other 
jurisdictions. 


A Centre for Health Economics and Policy Analysis study was commissioned 
by the Ministry to examine health care utilization and costs in health service 
organizations and fee-for-service practices. The Ministry will expedite this 
study and carefully analyze the results to determine if it has applicability to 
the Primary Care Network Program. 











CAPITATION RATES 


Under capitation funding, a physician is paid a fixed amount based on patient enrolment that 
does not vary with the volume of medical services provided to enrolled patients. 


The Ministry has calculated capitation rates using the amounts historically paid by OHIP for the 
services PCNs are required to provide to their enrolled members. These rates have been 
adjusted to reflect the average usage of services by patient age and sex. However, these rates 
have not been adjusted for other factors that may affect the need for primary health care, such 
as patients’ medical histories. Therefore, physicians with patients that have a greater-than- 
average need for primary health care services may be disadvantaged when compared to 
physicians whose patients have a lower-than-average need. Physicians that are part of a PCN 
are required to enrol patients regardless of their health status. However, if capitation rates are: 
not adequate to compensate physicians for providing care for a greater-than-average number of 
high-care patients, physicians with such caseloads may be motivated to stay with, or return to, 
the current fee-for-service system. 


Capitation rates that accurately reflect the services required for a PCN physician’s enrolled 
patients are a critical factor in the success of the PCN Program. Rates that are too low may 
discourage physicians from participating in the Program and rates that are too high could make 
PCNs unnecessarily expensive to fund. The Ministry has indicated that it intends to review 
capitation rates after the first eighteen months that PCNs have been in operation. 


During our audit, we contacted experts in health services research and reviewed developments 
in other jurisdictions to determine the feasibility of adjusting capitation rates to more fully reflect 
the primary care health needs of patients. While all the available adjustment methods had 
limitations, there were several that warranted consideration by the Ministry. One of these was 
the Adjusted Clinical Group system being used by the Province of British Columbia in its 
Primary Care Demonstration Project. 
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Recommendation 


To help ensure that funding of primary care networks is equitable, the 
Ministry should consider options, including those utilized by other 
jurisdictions, for adjusting capitation rates to equitably reflect the level of 
services being provided. 


Ministry Response 


The Ministry is examining additional means for adjusting the primary care 
reform capitation rate beyond age and gender. Studies are underway to 
examine the applicability in Ontario of the Adjusted Clinical Group Case-Mix 
Adjustment system. In addition, researchers at the Centre for Health 
_Economics and Policy Analysis will be providing advice and 
| recommendations on the potential to adjust current capitation rates using 
modifiers such as rural vs. urban practices and specific chronic disease 
indices. The Ministry is committed to examining means of further diversifying 
the form of funding to providers so that remuneration will consist of 
capitation, fee-for-service for encouraged services and additional bonus 
_ incentives for population health measures. 


REFERRALS TO SPECIALISTS 


Appropriately referring patients to specialists is important for achieving cost-effective health 
care. Unnecessary referrals increase health care costs and may reduce access to scarce 
resources, while failing to refer patients may result in poorer health outcomes and may increase 
health care costs in the long run. 


Experience in one jurisdiction suggested that using capitation to fund primary health care 
generally resulted in higher referral rates to specialists. However, one of the medical experts 
we consulted suggested that the current fee-for-service system may act as an incentive for 
some physicians to refer high-care patients to specialists, thus developing a practice that serves 
patients with simple health care requirements. 


The importance of the relationship between primary care physicians and specialists has been 
recognized by major Canadian medical organizations representing both family and specialty 
physicians. It is also important to know whether one method of funding primary health care 
carries a greater risk of inappropriate referrals than another method of funding. However, | 
higher referral rates may not necessarily indicate the appropriateness of referrals. The Ministry 
and health care providers need to have adequate information about how well this relationship 1s 


working and how it can be improved. 
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Recommendation 


To help ensure that the process for referring patients to specialists is cost 
effective, the Ministry should: 


e determine whether different methods of funding primary health care 
affect referrals to specialists; and 


¢ develop methods for monitoring and improving the quality of the referral 
process. 





Ministry Response 


The Ministry will undertake to measure the referral patterns of fee-for-service 
and primary care reform physicians in order to determine if different methods 
of funding primary health care affect referrals to specialists. The Ministry has 
asked the University of Toronto to provide advice and recommendations on 
the use of referral rates as a performance measure for primary care networks. 

















INTERDISCIPLINARY PRIMARY HEALTH CARE 


Primary health care providers include physicians, nurse practitioners, midwives, dietitians and 
others. Primary health care is more cost effective if the most appropriate providers deliver care 
to patients. The most appropriate providers are those who can deliver the health care required 
for the least cost at a quality equal to or better than that of other providers. 


For example, some studies in other jurisdictions have suggested that it may be more cost 
effective for nurse practitioners to provide many of the services traditionally provided by 
physicians. Physicians working in a team with nurse practitioners could deal with more complex 
cases and be available for consultation. 


For PCNs to deliver high-quality care as cost effectively as possible, the Ministry needs to 
determine the combinations of primary health care providers that are most appropriate in 
various circumstances. These combinations are likely to vary somewhat with the characteristics 
of different patient rosters. If appropriate combinations are encouraged, both taxpayers and 
patients will benefit. 





Recommendation 


To help ensure that primary health networks deliver cost-effective primary 
health care, the Ministry should: 


* thoroughly evaluate the experiences of other jurisdictions to identify best 
practices that warrant consideration in Ontario; and 


* provide guidance on the combinations of health care providers that are 
required to provide high-quality and cost-effective health care. 
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Ministry Response 









The Ministry will continue to evaluate the experiences of other jurisdictions to 
identify best practices that warrant consideration in Ontario. The Ministry 
now funds seven nurse practitioners working in the primary care reform 
pilots. Consultations with other provider groups and stakeholders will occur 
as the Ministry explores different combinations of health care providers that 
might be required to provide high-quality and cost-effective health care. 





ROSTERLIMITSAND VERIFICATION 


Under current agreements between the Ministry and PCNs, roster limits apply only to PCNs 
that are funded on a capitation basis. There are no roster limits for PCNs that select reformed 
fee-for-service as their method of payment. At the time of our audit, with some exceptions, 
PCNs paid by capitation were allowed to roster a maximum number of 2,200 patients for each 
full-time PCN physician. 


The four-year agreement between the Ministry and the Ontario Medical Association, which 
was effective April 1, 2000, states that “pending an evaluation to the contrary, no limit shall be 
set on roster sizes in future primary care network (PCN) contracts, provided that the physician 
to whom the patient is rostered personally and directly provides the majority of primary care 
medical services to the patient.” 


Roster limits may help ensure that patients have reasonable access to their primary care health 
providers and that providers have the time necessary to provide patients with high-quality care. 
A reasonable maximum roster size depends on a number of factors, including the number and 
type of primary care health providers and the health care needs of the rostered patients. 


Some health care experts have suggested that the health needs of rostered patients could be 
measured using an index that rates the complexity of the care required for each patient. The 
index could then be used to establish roster sizes for individual PCN physicians. 


In establishing roster limits, other factors should also be considered, such as the extent to which 
providers serve non-members and ease of administration. Although we found no formal 
research studies dealing with roster limits, health services experts we consulted indicated that 
such research would be practical. 


As we explained earlier in this report in connection with HSOs, because PCNs are generally 
funded based on the number of patients on their rosters, the Ministry will also need to ensure 
that PCN rosters are accurate. However, while the agreements permit the Ministry to request 
that PCNs verify up to 5% of their enrolled members, there is no provision for additional 
verification. 





Recommendation 







To help ensure that roster limits are reasonable, the Ministry should 
research best practices in other jurisdictions and establish a sound basis for 


setting Primary Care Network (PCN) roster limits. 
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The Ministry should also ensure that an effective verification process is 
implemented for PCN patient rosters. 


Ministry Response 


The agreement between the Ministry and the Ontario Medical Association 
(OMA) indicates that, subject to an evaluation to the contrary, there are to be 
no roster limits in the rollout of primary care networks across the province. 
The Ministry will endeavour to revisit the means by which roster limits are 
arrived at in other jurisdictions and implement measures appropriate to 
Ontario’s reform where possible following discussions with the OMA. 





The Ministry will annually verify 5% of physicians’ rosters. Based on the 
results of the evaluation, the 5% roster verification may be revised in 
subsequent agreements with primary care networks. 














PERFORMANCE MEASUREMENT AND REPORTING 


In its December 1999 report, Primary Health Care Strategy, the Health Services 
Restructuring Commission recommended that primary care group practices, as part of their 
accountability to their patients and the Ministry, submit regular report cards addressing client 
care, human resources management and financial responsibility. 


As mentioned earlier in this report, the Ministry intends to evaluate the PCNs. However, it also 
needs to establish ongoing performance measures that cover the major aspects of care. These 
include health outcomes, patient access to health care services and patient satisfaction. 


However, there is a risk that inadequately designed performance measures may lead to 
undesirable actions. For example, if a patient’s prior health status is not properly taken into 
account when assessing performance, physicians may be less willing to care for sicker patients 
because such patients may negatively impact on the assessment of their performance. 


Performance measurement and reporting requires valid and reliable information. Information 
that is not collected uniformly or is incorrect or incomplete cannot be used to make meaningful 
comparisons. The Ministry currently relies on its OHIP billing system to collect information 
about the services provided to patients by PCN physicians who record, but do not bill for, their 
services on the system. However, the Ministry has no means of ensuring that the information it 
receives is accurate or complete. Further, the system was not designed to capture health 
performance information or information about services provided by other health care 
professionals, such as nurse practitioners. 








Recommendation 


To enable it to assess the quality of care provided by primary care networks 
(PCNs), the Ministry, in collaboration with primary health care providers, 
should develop: 


* appropriate performance measures and standards; and 
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_ other research projects. 
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2 health performance information system that meets the needs of the 
_ Ministry and PCNs. 


Ministry Response 


Work to develop performance measurements is already underway with the 


University of Toronto. The research objective is to develop a framework for 


assessing performance in primary care. 


The Ministry will undertake to examine the feasibility of a health performance _ 
information system that could draw for its development on the performance | 
measures and standards identified in the University of Toronto’s report and 
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MINISTRY OF HEALTH AND LONG-TERM CARE 


Ontario Midwifery Program 





BACKGROUND 


The Ministry of Health and Long-Term Care’s Community and Health Promotion Branch is 
responsible for administering and funding the Ontario Midwifery Program. 


On December 31, 1993, midwifery became a regulated health profession in Ontario. The 
Midwifery Act defines the practice of midwifery as the assessment and monitoring of women 
during pregnancy, labour and the post-partum period and the provision of care to women and 
their babies during normal pregnancy, labour and post-partum period. 


The Ontario Midwifery Program was established in 1994 to fund professional midwifery 
services. Based on information provided by midwifery practice groups, the Ministry estimated 
that midwives attended approximately 3,800 births in the 1998/99 fiscal year. By the 2003/04 
fiscal year, the Ministry expects that midwives will be attending about 12,000 births annually in 
the province. 


For the 1999/2000 fiscal year, the Ministry provided approximately $17 million to fund the 
provision of midwifery services. 


AUDIT OBJECTIVES AND SCOPE 


The objectives of our audit of the Ontario Midwifery Program were to assess whether the 
Ministry had adequate procedures in place: 


* — to ensure that the Program was managed with due regard for economy and efficiency; and 
* to measure and report on the effectiveness of the Program. 


Our audit was performed in accordance with the standards for assurance engagements, 
encompassing value for money and compliance, established by the Canadian Institute of 
Chartered Accountants and accordingly included such tests and other procedures as we 
considered necessary in the circumstances. Prior to the commencement of our audit, we 
identified the audit criteria that would be used to address our audit objectives. These were 
reviewed and agreed to by senior ministry management. 


In conducting our audit, we reviewed and analyzed program policies and procedures, 
interviewed ministry staff and met with representatives of the College of Midwives of Ontario 
and the Association of Ontario Midwives. We were unable to rely on the Ministry’s Internal 
Audit Service to reduce our audit work since it had not issued any recent relevant reports on 
the Program. Our audit covered the period up to May 2000. 
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OVERALL AUDIT CONCLUSIONS 


The Ministry had developed a new funding agreement for midwifery practice groups that meets 
the essential requirements of Management Board of Cabinet’s Directive on Transfer Payment 
Accountability. However, the Ministry had not yet developed the procedures necessary to 
ensure that the expanding Ontario Midwifery Program is being managed with due regard for 
economy and efficiency and to measure and report on whether the Program is meeting its goals 
and objectives. Specifically, we found that the Ministry had not: 


* assessed whether the current delivery and funding model for midwifery services was cost 
effective; 


* instituted a process to collect and analyze information needed to evaluate the success of the 
Program; 


¢ defined greater equity of access to midwifery services; and 


* assessed the appropriateness of the process used to refer midwifery clients to specialists in 
non-emergency situations. 


DETAILED AUDIT OBSERVATIONS 


In 1987, the Task Force on the Implementation of Midwifery in Ontario recommended to the 
Minister of Health that midwifery be established as a regulated health profession. In 1993, the 
Midwifery Act was proclaimed, and midwifery became a regulated health profession in 
Ontario. Under the Regulated Health Professions Act, the College of Midwives of Ontario 
was established as the governing body for midwifery in Ontario. The College’s responsibilities 
include: 


¢ regulating the practice of the profession and governing the members in accordance with 
legislation, regulations and by-laws; 


¢ developing, establishing and maintaining standards of qualification for midwives; and 


¢ developing, establishing and maintaining programs and standards of practice to assure the 
quality of the practice of the profession. 


In 1994, the Ontario Midwifery Program was established to fund midwifery services with the 
goals of improving maternal and child outcomes and providing choice in maternity care through 
managed, community-based midwifery services. The Program’s objectives include ensuring: 


* consumer involvement in the planning, delivery and evaluation of services, 
* greater equity in access to midwifery services across Ontario; and 


* an equitable funding mechanism that supports the integration of midwifery services into the 


funded health care system. 


From 1994 to 1999, on an interim basis, the Ministry had contracted with and funded a non- 


in turn had agreements with midwifery practice groups to provide services 


rofit agency, which ous 
3 Pe p of two to five midwives. 


in specific geographic areas. Practice groups are usually made u 
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In 1999, the Ministry approved 20 non-profit agencies that, effective April 1, 2000, replaced the 
previous single non-profit agency. As part of the transfer to 20 agencies, the Ministry developed 
a new funding agreement for use by these agencies and their practice groups that meets the 
essential requirements of Management Board of Cabinet’s Directive on Transfer Payment 
Accountability. In January 2000 there were 39 midwifery practice groups and approximately 
180 registered midwives. The Ministry expects the number of registered midwives to grow by 
40 to 50 a year as new graduates enter the profession. 


Most of the funding a practice group receives is based upon the number of billable courses of 
care provided. A billable course of care usually refers to the midwifery services provided to a 
client. However, a billable course of care may also include other activities, such as outreach 
activities directed to populations with special health care needs. Billable courses of care that do 
not involve the provision of midwifery services are subject to specific conditions and limitations 
and must be approved by the Ministry in advance. 


The fee for a billable course of care includes a fixed component of $575 to cover overhead 
costs and a professional component that varies with the midwife’s number of years of 
experience. For example, the professional component for a midwife with five years experience 
is $1,575. Certain expenses, such as travel, are reimbursed separately. 


ASSESSING COST EFFECTIVENESS 


Information about cost effectiveness is important in assessing the funding and organization of 
midwifery services since these arrangements affect the cost of delivering these services. 


In 1996, the Ministry, the College of Midwives of Ontario and the Association of Ontario 
Midwives identified the need to develop an Information Management and Technology Plan for 
the Program. Consultants were engaged and made recommendations for a shared information 
system that would meet the needs of all three parties. At the time of our audit, the Ministry had 
not acted on these recommendations. 


In 1997, the Ministry funded a study to make preliminary cost comparisons between midwifery 
care and family physician maternity care using health data from ministry systems, such as 
OHIP. However, the study was never completed. An evaluation of cost effectiveness would 
also have to address the quality of care provided, for instance, whether midwifery reduces the 
incidence of interventions such as cesarean sections and hospital stays when compared to 
physician-attended low-risk pregnancies. 


At the time of our audit, the Ministry did not have adequate information to effectively monitor 
and determine whether the objectives of the Midwifery Program were being met. While the 
Ministry received quarterly activity reports containing caseloads and number of births, it did not 
obtain sufficient information about the outcomes of midwifery services. 


Recognizing the need for data on the utilization of resources and outcomes, the Association of 
Ontario Midwives began to collect data from midwifery practice groups on a voluntary basis. 
Although data has been collected for 1998 and 1999, the Association has indicated that it has 
insufficient resources to compile and analyze it. 


Information about the quality and cost effectiveness of midwifery services is also an important 
element in health workforce planning. In a December 1999 report on physician resources in 
Ontario, a provincially appointed fact finder noted that physician workforce planning cannot 
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occur in isolation since it is influenced by many factors, including the role of other practitioners 
with overlapping scopes of practice. The fact finder also noted that future planning for and 
funding of midwifery needed to be integrated with overall planning for obstetrics services. 




























Recommendation 


To help assess the quality of midwifery services and assess whether these 
services are delivered efficiently and effectively, the Ministry, along with the 
College of Midwives of Ontario and the Association of Ontario Midwives, 
should: — 

* determine what information is needed to make these assessments; and 
-* ensure that the information is collected and analyzed. 


Ministry Response | 


The Ministry will work with the Association of Ontario Midwives and the 
College of Midwives of Ontario to develop an evaluation system and 
management information system to assist in the assessment of the efficiency 
and effectiveness of midwifery services delivered through the Midwifery 
Program. This type of health program effectiveness research is complex and 
requires extensive comparative data from other parts of the health care 
sector. Current limitations in the available data may affect the Ministry’s 
ability to make valid comparisons. 


Since 1996, the Ministry has been developing comprehensive funding 
agreements for midwifery practice groups and transfer payment agencies; 
revising the program’s reporting systems, policies and procedures; and 
devolving funding to 20 local transfer payment agencies. The Ministry will 
now embark on developing an evaluation tool for the Midwifery Program. As 
a first step, the Ministry will ensure that the evaluation and management 
information needs of all stakeholders are identified. The Ministry will then 
develop a system to gather data from midwifery service providers, aggregate 
it at the local and provincial levels and provide reports to midwifery 
stakeholder agencies and the public. 








EQUITABLEACCESS 


One of the objectives of the Midwifery Program is to ensure greater equity of access to 
midwifery services across Ontario. However, the Ministry has not defined * greater equity of 
access.” If decisions about access are to be made, the Ministry requires information about who 


has access to midwifery services now. 


As of April 1, 2000, the new funding agreement requires midwifery pra ! 
provide services to a minimum number of women referred to the group by their 
agency. This number is negotiated between each practice group and its agency. If ie 

agencies reserve for their clients a significant number of places within midwifery practice 


ctice groups to offer and 
local approved 
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groups, this decreases the number of spaces available to others. Ministry staff indicated that 
they would not be receiving copies of the agreements between the local agencies and their 
practice groups. Therefore, the Ministry will not be able to monitor or assess the impact of this 
process on access to midwifery services. 


In 1987, the Task Force on the Implementation of Midwifery in Ontario stated that midwifery 
services should be equally accessible to all women and recommended that midwives be 
prohibited from seeking or obtaining payment for midwifery services from their clients. While 
the agreements with midwifery practice groups and the College of Midwives of Ontario’s 
regulations prohibit midwives from accepting payment from a client for services paid for by the 
Ministry, midwives are not prohibited from having private clients. Therefore, it is possible that 
clients who are willing to pay more than the Ministry pays will have greater access to services. 


Unlike for physicians’ services, to receive publicly funded midwifery services, a woman does 
not need a valid health insurance number. However, to be eligible to obtain publicly funded 
midwifery services, a woman must be resident in the service area described in the funding 
agreement. The funding agreement simply defines a resident as “‘a female person who is a 
resident in the service area.” This contrasts with the detailed definition of “resident” used for 
the purposes of determining eligibility for other ministry programs such as OHIP. 








Recommendation 


To determine whether the Midwifery Program is meeting its objective of 

ensuring greater equity of access to midwifery services, the Ministry should: 

* clearly define the meaning of “greater equity of access”; 

* assess the impact of the allocation of midwifery services in the 
agreements between local agencies and midwifery practice groups; 

* review the arrangements that permit midwives to have private clients; 
and 


* clearly define “resident” for the purposes of eligibility for publicly funded 
midwifery services. 





Ministry Response 


Prior to 1994, midwifery services were available only to women and families 
who could afford to pay privately. One of the objectives in establishing a 
publicly funded midwifery program was to improve equity of access by 
ensuring that ability to pay was not a barrier to access. As the number of 
midwives increases, the service will be made accessible to more women in 
more locations across the province. The Ministry will assess whether the 
objective of a publicly funded midwifery program should be revised to more 
clearly reflect the intention. 
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As part of the Program’s mandate, funding has devolved from one central to | 

20 local transfer payment agencies. These agencies are well positioned to 
have a broad overview of the service needs of their local populations. The 

funding agreement requires agencies to negotiate with each practice group 
the number of clients the agency may refer to it in any year. The intent of this 

requirement is to create greater equity of access since the agency will refer 

|) women who it identifies as having special needs or who may otherwise have 
been unable to obtain midwifery care on the usual first-come first-served 
basis. The Ministry monitors the allocation of midwifery resources and their 

utilization throughout each year through quarterly reports provided to the 

| Ministry. 


Transfer payment agencies fund midwives in accordance with an agreement 
that establishes midwives as independent contractors. One important legal 
element inherent in this model is the ability of midwives to determine how 
_and from whom they derive their income. The Ministry will monitor whether 
this has any impact on access to midwifery services. 


The Ministry notes that there are many definitions of resident in different 
pieces of health care legislation that are used for many different purposes. 

The Ministry will review the residency definition to ensure that it meets the 
_needs of the Midwifery Program and therefore the population being served. 





REFERRALS TOSPECIALISTS 


The Task Force on the Implementation of Midwifery in Ontario recommended that the College 
of Midwives of Ontario develop standards of practice for midwives, including criteria for 
consultations with and referrals to physicians. Accordingly, the College developed guidelines 
requiring midwives to consult and transfer care to a physician under specific conditions. 
Midwives are also required to consult with a physician when specific health conditions are 
present during pregnancy, labour and birth or during a period of at least six weeks after birth. 


Since the inception of the Program, the College and the Association of Ontario Midwives have 
expressed concerns to the Ministry about the difficulties some midwives have experienced in 
directly consulting with medical specialists when, according to their professional standards of 
practice, consultation was required. Instead, midwifery clients generally needed to be referred 
to a specialist by a family physician. 


In its April 30, 1996 report on primary health care, the Provincial Coordinating Committee on 
Community and Academic Health Science Centre Relations recommended to the then Minister 
of Health that midwives be allowed to make referrals directly to some specialists, for example, 
obstetricians. 


In April 1999, the Ministry created two OHIP fee codes for emergency assessments of clients 


of midwives. We were informed that midwives continued to have difficulty directly neat ee 
specialists in non-emergency situations. In these situations, midwifery clients eee 
to be referred to a specialist by a family physician or hospital emergency department. Ihe 
current referral practice may create additional costs for the health care system and 
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inconvenience women and their children, such as requiring them to make unnecessary visits to 
family physicians and hospital emergency wards. 





Recommendation 










The Ministry should ensure that the current process for referring clients of 
midwives to specialists does not result in unnecessary visits to family 
physicians or hospital emergency departments. 





Ministry Response 


The family physician’s role in the medical system cannot be overestimated. 
They help ensure that patients receive the best possible care and reduce 
pressure on scarce specialist resources. Nonetheless, direct referrals from 
midwives to specialists are appropriate under certain circumstances. The 
Ministry agrees to raise the associated professional and payment policy 
issues with stakeholders within the College of Physicians and Surgeons of 
Ontario, the Ontario Medical Association, the College of Midwives of Ontario 
and the Association of Ontario Midwives and work toward a resolution. 
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MANAGEMENT BOARD SECRETARIAT 


Movable Assets 





BACKGROUND 


Ministries’ movable assets consist primarily of office furniture and equipment, such as 
photocopiers and fax machines; information technology (IT) equipment, including desktop and 
notebook computers, printers and other peripherals; audio-visual equipment, such as televisions, 
VCRs and cameras; and motor vehicles. 


During the 1998/99 fiscal year, all ministries spent approximately $500 million on movable 
assets, much of which was spent on IT equipment in preparation for Y2K. However, the total 
value, type and quantity of movable assets on hand were not known because ministries did not 
keep adequate records in that regard. 


Management Board of Cabinet has issued several directives related to the acquisition and 
management of equipment, supplies and services, including information technology resources. 
These directives include the following principles, which in turn are reflected in mandatory 
requirements. 


Principles 


* The overall objective is to acquire and supply at the right time and in the most economical 
manner the appropriate level of equipment, supplies and services required to meet 
government needs. 


¢ For IT procurements specifically, ministries are to acquire IT to fulfil identified business 
needs through a fair, open and competitive procurement process that treats suppliers fairly 
and equally. 


* Once acquired, all equipment, supplies and services are to be managed efficiently, 
effectively and economically. 


Mandatory Requirements 


* Ministries must ensure that planning forms an integral part of the acquisition process and 
includes clear definitions of requirements, justification for the acquisition, consideration of 
alternative ways to satisfy the need and selection of the most appropriate option for 
approval. 


¢ Except for certain equipment, supplies and services that must be obtained from approved 
mandatory central common services, acquisitions must be made through a competitive 
process that ensures the best value for funds expended to meet the specific needs and 
promotes fair dealings and equitable relationships with the private sector. 
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¢ Ministries must choose the supplier that meets all mandatory requirements and has the 
lowest evaluated cost. 


Once acquired, movable assets must be managed in an efficient, effective and economical 
manner. Appropriate systems must be established and maintained to ensure the effective 
management and security of these assets. To help maintain their security, the existence of 
movable assets must be periodically verified at intervals that vary with the nature of the assets, 
but in no case less than every four years. 


AUDIT OBJECTIVES AND SCOPE 


Our audit objectives were to assess whether government-wide and ministry-specific policies 
and procedures were adequate to ensure that movable assets were: 


* acquired based on assessed needs and in an economical manner; and 
* managed efficiently, effectively and economically. 


We conducted our audit work at five ministries: Consumer and Commercial Relations, 
Education, Health and Long-Term Care, the Solicitor General and Transportation. Since the 
Management Board Secretariat (MBS) develops government-wide procurement policy and had 
entered into government-wide standing agreements for IT equipment acquisitions that were 
used by most of the five ministries, we also reviewed the role of MBS in the procurement 
process. 


We issued detailed reports to each of the deputy ministers of the ministries included in our 
scope. Because MBS sets government-wide policies and standards for acquiring and managing 
movable assets, we summarized the more significant issues addressed in the individual reports 
to the ministries in this report to MBS. 


Our audit work with respect to movable asset acquisitions focused on the leasing of IT 
equipment, since that accounted for an overwhelming majority of all movable asset 
expenditures during the 1998/99 fiscal year. The scope of our audit included a review and 
analysis of relevant files and administrative procedures, inspection of a sample of movable 
assets, as well as interviews with appropriate ministry and MBS staff. 


Prior to the commencement of our work, we identified the audit criteria that would be used to 
address our audit objectives. These criteria were reviewed and agreed to by senior 
management of the five ministries audited and MBS. 


Our audit included the period up to March 2000 and was conducted in accordance with 
standards for assurance engagements, encompassing value for money and compliance, 
established by the Canadian Institute of Chartered Accountants, and accordingly included such 
tests and other procedures as we considered necessary in the circumstances. 


The work of the ministries’ internal audit services did not affect the extent of our audit work 
because they had not conducted any work on movable assets during the last three years. 
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OVERALL AUDIT CONCLUSIONS 


We concluded that, although the ministries generally followed the process recommended by 
MBS for acquiring IT equipment in that they used the MBS standing agreements for their 
equipment acquisitions, doing so did not ensure that they received value for money spent. In 
particular, we found that the ministries needed to work with MBS to ensure that: 


¢ IT equipment is acquired at a competitive price. 


* Assessments of the various makes and models of available equipment are made and 
documented to ensure selected equipment is the most appropriate and cost effective for 
their needs. 


¢ The desirability and cost-effectiveness of leasing IT equipment in comparison to other 
3 5 1 2 forms of financing or purchasing is established. 


¢ Equipment leases are entered into only when necessary and only on competitive terms. 
In order to comply with the MBS directives, the ministries also needed to: 

¢ maintain accurate and up-to-date listings of all owned and leased movable assets; 

¢ periodically verify the existence and efficient deployment of all movable assets; and 


¢ follow up on any missing or underutilized assets on a timely basis so that any necessary 
corrective action can be taken. 


DETAILED AUDIT OBSERVATIONS 


ACQUISITION OF INFORMATION TECHNOLOGY 
EQUIPMENT 


INFORMATION TECHNOLOGY EQUIPMENT PRICING 


In 1996, MBS established standing agreements for the acquisition of IT equipment with 22 
major equipment manufacturers. Under the terms of these agreements, ministries can 
determine a manufacturer’s product availability and pricing from Web sites that are updated 
periodically. Although ministry use of these standing agreements is optional, four of the five 
ministries included in the scope of our audit used them to acquire their IT equipment. The fifth 
ministry had established its own purchase arrangements with a supplier; however, the prices 
obtained from that supplier were generally equivalent to the prices offered under the 
manufacturer’s standing agreement with MBS. 


Since 1997, ministries have been advised by MBS staff in writing that they were exempted 
from using competitive acquisition procedures for IT equipment acquired under the MBS 
standing agreements. We found that ministries generally did not assess the pol bathe ) 
prices offered under the standing agreements because they believed that the ania | ie 
agreements were entered into competitively, as required by Management Board o : ay - 
directive. In addition, the need to negotiate additional discounts for large-volume purchases was 
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not well communicated by MBS or understood by the ministries. For example, while MBS staff 
advised us that ministries were expected to receive additional discounts for large-volume 
purchases, ministry staff we interviewed all assumed that such discounts were already 
reflected in the prices posted on the manufacturers’ Web sites. 


The process for establishing these standing agreements allowed all vendors the opportunity of 
meeting the mandatory technical requirements. However, pricing under the proposed 
agreements was not evaluated as required by the Management Board of Cabinet directive. 
Specifically: 


* In January 1996, MBS issued a request for quotations (RFQ) to qualified manufacturers for 
the supply of IT equipment through suppliers at preferred Government of Ontario prices. 
We noted that all responding manufacturers were awarded a standing agreement as long as 
they complied with the mandatory technical requirements. 


* Manufacturers were required to offer their equipment at preferred Government of Ontario 
prices. However, since the term “preferred Government of Ontario prices” was not 
defined, it could not be assumed to represent competitive prices, which change frequently 
over time. 


Our review of the RFQ and the resultant standing agreements with the manufacturers also 
indicated that: 


* The manufacturers were required to demonstrate how MBS would be assured, and be able 
to audit, that the manufacturer continues to offer products at preferred Government of 
Ontario prices. 


* Manufacturers were required to maintain the calculations used to establish their preferred 
Government of Ontario prices for a period of up to four years following the expiry or 
termination of their standing agreements and to make such records readily available for 
inspection by MBS staff. However, MBS staff had never asked for or reviewed any of 
these records to determine how preferred Government of Ontario prices were, in fact, 
established. 


At the conclusion of our field work, we noted that MBS had not periodically compared the 
prices in the standing agreements with other prices for comparable equipment to ensure that the 
standing agreement prices remained competitive. Subsequently, MBS indicated that they had 
recently compared their prices to those outlined in a U.S. study that illustrated prices a U.S. 
buyer would be willing to pay. MBS concluded that this indicated their prices were competitive. 
However, in our view this comparison was limited because it did not assess whether the prices 
being paid by the Ontario government were, in fact, comparable to prices being paid by other 
large Canadian public and private sector organizations. 


It was difficult to compare equipment prices available through the MBS standing agreements 
given the frequently unique features and model numbers identified on the suppliers’ Web sites. 
However, our comparison of the MBS standing agreement prices for the same or very similar 
equipment available from other sources indicated that the standing agreement prices were often 
not the most economical available. For example, in reviewing a sample of MBS standing 
agreement computer prices, we found that in most cases considerable savings could have been 
achieved if purchases had been made from sources other than those of the MBS standing 
agreements. 
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We also noted that the processes used by other jurisdictions to establish IT equipment prices 
under standing agreements differed significantly from the process followed by MBS. For 
example, the federal government and the provincial governments of Alberta and British 
Columbia ensured the continuance of competitive prices under their standing agreements by: 


* periodically retendering their agreements to establish new prices; 


* — establishing standing agreement prices based on a fixed percentage discount from the 
manufacturers’ published list prices; and 


¢ individually tendering large-volume purchases of equipment in excess of a threshold amount 
that varied with the type of equipment to be purchased. 


Subsequent to the completion of our audit fieldwork, we were advised by MBS staff that they 
were developing an IT procurement policy based on the Total Cost of Ownership (TCO) 
concept. The TCO concept is a comprehensive approach under which ministries will be 
required to assess the combined costs of IT equipment and services, such as configuration, 
installation, maintenance support and leasing, before making a purchase decision. 


To implement this approach to IT acquisitions, MBS intends to enter into a mandatory 
government-wide standing agreement with a vendor that will be selected through a competitive 
process by the end of fiscal year 2001/02. 


MBS also advised ministries that, as their own agreements with approved vendors expire, it will 
become mandatory for them to use the standing agreements established by MBS. It is therefore 
necessary for MBS to make substantial changes to the pricing provisions under its standing 
agreements. 
























Recommendation © 


To ensure that ministries acquire information technology (IT) equipment in 
compliance with Management Board of Cabinet directives, Management 
Board Secretariat (MBS) should work with ministries to ensure competitive 
| prices are achieved until a new competitive pricing structure is established 
| through implementation of the proposed Total Cost of Ownership concept. 


/ Management Board Response 


| MBS is developing a government-wide procurement mode! for desktop 
‘computers and will issue a request for proposals (RFP) by spring 2001 (which 
will involve the selection of a third-party service provider). The RFP will be to 
provide desktop equipment and services and will ensure pricing is in line with 
market conditions and ensure continuous benchmarking to provide best 


value for money spent. 


It should also be noted that as part of the information management and 
information technology (IM/AT) planning process, ministries were directed in 
1998, for the 1999/2000 IM/IT plans, to report on the Total Cost of Ownership 
(TCO) for desktop computers and to take steps to reduce the TCO ae 7 
desktop computers through standardization of equipment and software. 
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| well, though MBS began this process with desktop computers, MBS intends 
to look at all information technology expenditures from a TCO perspective. 






In addition, MBS will monitor and benchmark equipment prices on a quarterly 
basis and will continue its training program for staff involved in IT product 
procurement and in the use of IT-related standing agreements. 











INFORMATION TECHNOLOGY EQUIPMENT SELECTION 


Although ministries are expected to assess the available equipment models offered by the 
various manufacturers and select those that are the most appropriate and cost effective for 
their needs, we often found no evidence that this occurred. 


We understand that, in practice, some ministries selected equipment manufacturers based either 
on the equipment that was already deployed in their ministries or on the equipment used by 
other associated ministries. 


Similarly, we generally found no evidence to indicate that ministries had evaluated the various 
models and prices of available equipment to ensure that the models selected were the most 
appropriate and cost effective for their needs. We understand that the ministries often acquired 
equipment models from those available under the standing agreements based on 
recommendations from manufacturers or suppliers. 


As a result, the equipment acquired by ministries was not always the most appropriate or cost 
effective for their needs. For example, we noted the following: 


* After the Ministry of Education had selected equipment from a particular manufacturer to 
be acquired in preparation for Y2K, another equipment manufacturer requested that the 
Ministry evaluate the cost-effectiveness of its equipment. Although the Ministry found that 
for its requirement of comparable desktop computers alone, the other manufacturer’s 
equipment would have cost approximately $690,000 less over the term of a three-year 
lease, the Ministry nevertheless stayed with the manufacturer it originally selected. 


*  Staffat the Ministry of Health and Long-Term Care determined that the Ministry had paid 
approximately $800 more per desktop computer than it would have paid for a comparable 
model from the same vendor that would have also met their needs. 


Based on the results of our audit work in this area, we recommended in our individual reports to 
the ministries we audited that, in the future, the ministries should ensure they receive value for 
money spent by assessing the equipment models available from the various manufacturers and 
selecting for acquisition those that are the most appropriate and cost-effective for their needs. 


The ministries accepted the necessity of receiving the best value for money in the IT acquisition 
process. Specific manufacturers and models were often selected based on equipment already in 
place and acquired at prices available under the MBS standing agreements, which the ministries 
believed were entered into competitively. 


Ministries also indicated that they would consult and work with MBS to ensure that value for 
money is received for all future IT acquisitions. 
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INFORMATION TECHNOLOGY EQUIPMENT LEASING 


Management Board of Cabinet Directive 7-4 requires that ministries consider alternative 
financing arrangements for IT procurements exceeding $1 million. However, in October 1998 
MBS staff advised ministries of a change in policy direction requiring them to lease-finance 
their IT equipment, without having requested that Management Board of Cabinet amend 
Directive 7-4 accordingly. Buying or building their own equipment was recommended only for 
relatively rare circumstances, such as when the equipment required is unique, is unavailable 
through lessors or has an extraordinarily long life-span. 


In October 1996, MBS entered into a two-year master lease agreement with a leasing company 
for the leasing of IT equipment by ministries, agencies and other members of the broader public 
sector. Although ministry use of this agreement was optional, ministries were encouraged by 
MBS to lease their required IT equipment because doing so would enable costs to be spread 
over the useful lifetime of the equipment and result in predictable annual costs linked to ongoing 
business requirements. 


Given this change in MBS direction toward leasing, we found that all of the ministries we 
audited lease-financed most of their equipment requirements. Three of them used the MBS 
standing agreement while the other two established their own leasing agreements with different 
vendors. 


We noted that, in February 1998, Management Board of Cabinet approved the implementation 
of the government’s Information and Information Technology Strategy. As part of MBS’s 
presentation to Management Board of Cabinet detailing this Strategy, MBS specifically stated 
that IT equipment would be leased. However, neither MBS staff nor the ministries were able to 
provide us with an analysis that compared the cost-effectiveness of leasing in general or under 
the MBS standing agreement in particular to other forms of financing and that demonstrated 
that leasing was the most advantageous choice. Instead, most ministry staff advised us that 
they leased their equipment in order to comply with the most recent MBS policy direction and 
that cost was not a factor in their decisions. 


Our review of a number of leases in each ministry led to concerns as to whether value for 
money had been received in many cases. In particular, we noted that annual lease obligations 
were frequently prepaid, which calls into question the need to enter into any lease-financing 
arrangements at all. For example: 


¢ In March 1999, the Ministry of Education entered into a three-year equipment lease 
agreement under the MBS standing agreement and prepaid the full cost, which amounted to 
$17.2 million. By prepaying the entire amount at the beginning of the lease, the Ministry 
received a discount of $569,600 or approximately 3%, amortized over the three-year term 
of the lease. We noted that this discount compared very unfavourably to the 7% interest, or 
$1,250,000, charged by the leasing company and included in the prepaid value of the lease. 


In addition, since the prepayment was based on an over-estimation of its equipment 
requirements, at the time of our audit in November 1999, the leasing company was still 
holding over $3 million of the Ministry’s prepayment even though the Ministry S equipment 
requirements had been met. We understand that the Ministry was to have received interest 
equivalent to the three-month T-Bill rate on the outstanding prepayment balance. However, 
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in early 2000, despite numerous requests, the Ministry had not yet received any information 
from the leasing company on the amount of interest earned. 


* Similarly, the Ministry of Transportation prepaid a portion of its 1999/2000 lease obligation 
at the end of its previous fiscal year in order to use available funding for that year. As a 
result of this prepayment, the Ministry received a discount of $20,700 or approximately 
3.25%. We noted that this discount compared very unfavourably to the approximately 
7.25% interest, or $46,200, charged by the lessor and included in the year’s payment 
amount. 


The 1996 MBS standing lease agreement contained a provision for annual renewals at the 
conclusion of the original two-year term. In November 1999, the agreement was renewed, and, 
among other changes, it contained revisions that significantly reduced the residual values 
assigned to leased equipment from 17% to 7% on a three-year lease. This significantly 
increased the government’s leasing costs under the agreement. In addition, the leasing company 
added a provision to the agreement that allowed it to adjust the interest rates and residual 
values to be applied to leases entered into, although MBS may terminate the agreement on 90 
days’ written notice. 


Given these significant changes in terms and the resultant increase in leasing costs as compared 
to the original standing agreement, it is our view that the entire agreement ought to have been 
competitively retendered to ensure that it resulted in the government receiving the most 
competitive terms possible. 


Based on our findings, we recommended in the individual reports to the ministries we audited 
that, to help ensure IT acquisitions are financed in an economical manner, ministries should: 


* consider alternative financing arrangements for acquisitions valued at more than $1 million 
to ensure that they obtain competitive terms, as required by Management Board of Cabinet 
directive; and 


* — enter into a lease or other financing arrangement only when it is beneficial to do so. 


Ministries generally indicated that MBS had issued instructions that most IT assets be leased. 
They stated that they were adhering to those instructions. 


Ministries also indicated that they would consult and work with MBS to ensure that value for 
money is received in their future IT equipment leases. 





Recommendation 


To help ensure that information technology (IT) acquisitions are financed in 
an economical manner, as required by Management Board of Cabinet 
directive, Management Board Secretariat should: 


* formally assess the desirability and cost-effectiveness of leasing IT 
equipment in comparison to other forms of financing or purchasing; and 


* follow a competitive process when renewing its standing agreement for 
lease-financing services. 
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Management Board Response 























Leasing was recommended and endorsed in 1998 as a best practice by the 
Industry Sector Panel (comprising representatives from leading private sector 
IT practitioners, IT industry associations, the broader public sector and 
academia), which provided advice to the government during the strategy 
development process. Leasing as a best practice was subsequently 
incorporated into Management Board Secretariat’s (MBS) submission to the 
Management Board of Cabinet for approval for the Information and 
Information Technology Strategy. However, in order to ensure that this 
continues to be an appropriate strategy, MBS will periodically reassess the 
advantages and cost-effectiveness of leasing and alternate models of 
financing IT equipment, including desktop computers, in relation to 
supporting the Information and Information Technology Strategy. 


In that context, MBS will undertake a competitive process to establish a new 
standing agreement for lease-financing services. However, it is important to 
note that the existing MBS leasing agreement was extended for an additional 
year in the context of a planned request for proposal (RFP) for desktop 
equipment and services and to ensure that a leasing vendor was available to 


allow ministries to finance equipment acquisitions to meet critical | 





requirements. 








MANAGING MOVABLE ASSETS 


In order to safeguard ministry investments in all movable assets, Management Board of Cabinet 
directives require ministries to establish and maintain appropriate systems of internal control to 
ensure the effective management and security of assets. Requirements of an effective system 
include: 


¢ clear definitions of movable assets, including threshold amounts above which movable 
assets should be tracked, and a permanent record of assets by location; 


¢ identification of specific security requirements; 
¢ physical verification of assets at least once every four years, 
° identification and follow-up of missing or underutilized items; and 


¢ clearly assigned accountability and responsibility. 


All five of the ministries we audited had maintained a centralized movable asset tracking 
system in the past. However we were advised by three of the ministries that they had 
discontinued the use of these systems as a result of administrative downsizing. In these cases, 
individual ministry branches were subsequently expected to track and account for the movable 
assets assigned to them. The other two ministries continued to maintain their centralized 
movable asset tracking systems, although individual ministry branches were responsible for 
maintaining the accuracy of the information on these systems. 


ovable assets indicated that the individual ministry 


e systems for non-IT m 
oat ck them, the asset 


branches either did not track these assets at all, or, if the branches did tra 


215 


Special Report: Accountability and Value for Money 





listings were inaccurate. As a result, the ministries were not in a position to determine non-IT 
movable assets on hand or the extent of missing or underutilized items. 


With the onset of concerns about Y2K, ministries recognized a need to more effectively 
monitor their [T equipment, and as a result, the majority of them again began to track their IT 
equipment centrally in 1999. We tested the accuracy of the IT asset listings for the items that 
were tracked and found that these listings were generally accurate and complete. However, we 
did note some omissions from the ministries’ tracking of IT equipment. For example, IT 
equipment that was not connected to a ministry’s network, ministry-owned equipment and 
unassigned equipment were often not tracked at all. 


However, given that the ministries only started to again track most of their IT equipment as of 
1999, they were generally unable to determine the extent of missing or underutilized items prior 
to that time. Only as leases expired or were otherwise terminated and the leased equipment had 
to be returned did the extent of missing equipment become clear. For example, our review of a 
sample of lease expirations at two ministries indicated the following missing items: 


Missing Leased Equipment 


Ministry of Education 


Total Units Percentage of 


Desktop Computers 











Notebook Computers 
Monitors 
Printers 











Source: Ministry of Education data 






Ministry of Solicitor General 
Total Units Percentage of 
Desktop Computers 2 as 
Notebook Computers i 


















Source: Ministry of the Solicitor General data 


Based on our audit work, we recommended in the individual reports to the ministries we 
audited, that, to help ensure that all movable assets are managed efficiently, effectively and 
economically, the ministries should: 


* identify and maintain up-to-date records of all owned and leased movable assets; 


* periodically verify the existence of all movable assets; and 
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* identify missing or underutilized items on a timely basis and take an 


1 y necessary corrective 
action. 


Ministries agreed that asset records should be maintained and periodically verified for 
completeness and accuracy. They also indicated that the introduction of the government’s 
Integrated Financial Information System will provide a cost-effective solution for this issue. 


SAFEGUARDING MOVABLE ASSETS 


In March 1998, Management Board of Cabinet issued an Information and Information 
Technology Security Directive that requires all IT resources be identified and assigned to an 


appropriate manager for accountability purposes, a security plan be maintained and a periodic 
review of the plan be conducted. 


Three of the ministries we audited had a movable asset security policy or guideline in place that 
provided ministry staff with information on how to ensure the physical security of their assets. 
The other two ministries did not have a formal security plan or documented security policies. 


We also noted a number of common security deficiencies as follows: 


* Inanumber of cases, access doors to ministry offices did not have locks and therefore 
could not be secured. In some cases where doors had locks, they were continuously 
propped open, which defeated the purpose of the locks. 


¢ When employees leave a ministry they must complete a clearance certificate to indicate 
that any ministry assets assigned to them have been returned. However, we found that 
these forms were often not completed or not submitted. Furthermore, even when they were 
submitted as required, the forms were designed in a way that made it unclear as to whether 
or not all assigned assets had been returned. 


Based on our audit work, we recommended in the individual reports to the ministries we audited 
that, to help safeguard their assets, the ministries should: 


* implement an appropriate security plan as envisioned by the Management Board 
Information and Information Technology Security Directive; and 


¢ — ensure that employees leaving a ministry return all assigned movable assets and that such 
returns are appropriately documented. 


Where specific security deficiencies were identified, the ministries agreed to take the necessary 
corrective action. In one case, a ministry indicated it had already done so. The ministries also 
indicated that they would implement the necessary policies and procedures to ensure that all 
assigned movable assets are returned when an employee leaves. 


SURPLUS MOVABLEASSETS 


With the exception of motor vehicles that are disposed of by the Ministry of Transportation, the 
MBS Shared Service Bureau is to dispose of all ministry movable assets by invitational tender 
or auction. In addition, under certain circumstances, computer equipment may be donated to 
schools and libraries under a program sponsored by the Ministry of Education, Industry Canada 


and private sector partners. 
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In light of the current freeze on purchasing furniture and the predominance of leasing 
information technology assets, the amount of assets surplused during the last several years is 
not thought to be significant. As a result, we focused our examination on the Ministry of 
Transportation’s procedures for disposing of surplus vehicles. 


DISPOSAL OF SURPLUS VEHICLES 


During calendar year 1999, approximately 2,400 government vehicles and other pieces of 
equipment were disposed of, generating gross proceeds of $4.4 million. Through a competitive 
process undertaken in 1996, the Ministry of Transportation retained the services of an auction 
group to dispose of the vehicles. Under the terms of its agreement, the auction group may 
dispose of vehicles through wholesale or public auction or salvage tender. However, in practice, 
most vehicles are disposed of by public auction, as required by current ministry policy. 


After the completion of each auction, a summary report is to be sent to the Ministry of 
Transportation detailing the sale price for each vehicle and the related commission, clean-up 
and transportation costs as well as any other miscellaneous expenses. We reviewed a sample 
of vehicle disposals and summary auction reports and noted the following: 


* We found that in 1999, 116 vehicles were sold for less than the expenses incurred and 
charged for their sale. For example, a 1984 Chevrolet Celebrity sold for $24 while the 
expenses charged were $265, resulting in the Ministry owing the auction group $241 for the 
vehicle’s disposal. 


Ministry staff advised us that the auction group had agreed to not charge expenses in 
excess of sales proceeds. However this practice was not being enforced. As a result of our 
audit, the Ministry has requested and received $41,000 from the auction group for 314 
vehicles and equipment sales where expenses exceeded sales proceeds, some dating back 
to 1996. 


* Although we noted that ministries are exempt from paying GST, nevertheless the Ministry 
paid $42,700 for GST on auction-related expenditures. 


* In 1999, the auction group also charged over $15,000 in miscellaneous expenses. The 
Ministry was unable to determine the nature or validity of these charges. 


We also found that the Ministry of Transportation did not ensure that all vehicles sent to the 
auction group were sold and reported as sold on a timely basis. In that regard we noted that, 
although the Ministry’s fleet management system is capable of producing reports identifying the 
number of vehicles sent to auction but not yet reported as sold, this function was not used. 


We requested this report and found that vehicles sent to the auction group as long ago as 1997 
had not yet been reported as sold by March 2000. Further we noted a risk that the vehicles may 
have been sold and the proceeds not remitted to the Ministry. 


In addition, under the terms of its agreement, the auction group is to remit the net proceeds of 
all sales to the Ministry within 15 days of an auction. However, we reviewed a number of 
auction reports and found that proceeds were often received late, ranging from 5 months to 21 
months after the date of the auction. 


As a result of our audit work in this area, we recommended that, to receive fair and timely 
value for vehicles disposed of, the Ministry of Transportation should ensure that: 
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* — expenses incurred for disposals, in accordance with the terms of the disposal agreement, do 
not exceed sales proceeds and exclude GST: 


* vehicles sent to auction but not reported sold within a reasonable period of time receive 
periodic follow-up; and 


* net proceeds are received promptly, in accordance with the terms of the disposal 
agreement. 


The Ministry agreed with the recommendation and acknowledged that there is value in 
increasing the level of contract monitoring. The Ministry intends to develop and implement 
additional administrative procedures for monitoring to ensure the timeliness of vehicle disposal 
and remittance of proceeds, in accordance with contract terms. 
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MINISTRY OF NATURAL RESOURCES 


Forest Management 
Program 





BACKGROUND 


The mandate of the Ministry of Natural Resources is the sustainable development of the 
province’s natural resources, including the economies and communities that depend on these 
resources. One of these resources is Ontario’s 700,000 square kilometres of forested land, of 
which 44% is classified as productive Crown forest. 


The Ministry estimates that 40 communities and 90,000 jobs are dependent on the Ontario 
forest industry. In 1997, the industry shipped $4.5 billion worth of wood products and $9.1 billion 
worth of paper and related products. 


Under the Crown Forest Sustainability Act, the Ministry is responsible for ensuring the long- 
term health of Ontario’s Crown forests. The Act provides for the regulation of forestry 
planning, information, harvesting, renewal, trust funds for reforestation, and processing facilities, 
such as pulp and saw mills. The legislation also includes sanctions and penalties for non- 
compliance. 


With the enactment of the Crown Forest Sustainability Act in 1994 and the implementation of 
a ministry downsizing and reengineering plan in 1996, licensed forest management companies, 
rather than the Ministry, became directly responsible for forest sustainability planning and were 
required to carry out forest renewal on behalf of the Crown. The Ministry’s role in ensuring the 
long-term health of Crown forests has progressively become one of overseeing the activities of 
these companies. For the purposes of forest harvesting and renewal, most of the province’s 
productive Crown forests were divided into 68 units, most of which are managed by forestry 
companies under the terms of a sustainable forest licence granted by the Ministry. 


The Ministry’s oversight functions are carried out through its Forests Division and its Field 
Services Division, with three regional and 25 district offices. The Ministry’s Science and 
Information Resources Division also provides the forest management program with scientific 
knowledge, information management and information technology. 


In the 1999/2000 fiscal year, the Ministry spent $70.8 million on forest management. In addition, 
under the Crown Forest Sustainability Act, the Forest Renewal Trust and Forestry Futures 
Trust were set up to fund renewal expenditures made by licensed forest management 
companies and the Ministry. During the 1998/99 fiscal year, the two trusts paid out 

$104.8 million to licensed forest management companies for forest renewal and related 
activities. All licensees harvesting Crown timber pay forest renewal charges into the trusts. 
Forest renewal charges are based on a prescribed rate per cubic metre of timber harvested. 
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Forest management companies also pay Crown stumpage charges into the province’s 
Consolidated Revenue Fund. Stumpage charges are levied according to volume and species of 
wood cut. For the 1999/2000 fiscal year, stumpage charges totalled $155.7 million. 


AUDIT OBJECTIVES AND SCOPE 


The objectives of our audit of the forest management program were to assess whether the 
Ministry had adequate systems, policies and procedures in place: 


* to measure and report on the effectiveness of the program in ensuring the long-term health 
of Ontario’s Crown forests; 


* to ensure compliance with legislation and ministry policies and to identify areas requiring 
corrective action; and 


* — to ensure that ministry resources were managed with due regard for economy and 
efficiency. 


The criteria used to assess the program were discussed with and agreed to by ministry 
management. These criteria included: clearly defined objectives for forest sustainability; the 
collection of appropriate forest management information; adequate planning, monitoring and 
enforcement; and the accuracy and completeness of the revenue systems. 


Our audit was performed in accordance with the standards for assurance engagements, 
encompassing value for money and compliance, established by the Canadian Institute of 
Chartered Accountants, and accordingly included such tests and other procedures as we 
considered necessary in the circumstances. 


The scope of our audit, which covered the period up to March 2000, included discussions with 
ministry staff, a review and analysis of documentation at the Ministry’s head office in Sault Ste. 
Marie and visits to two regional and seven district offices. We also sent a questionnaire to each 
of the district offices we did not visit. 


The Ministry’s internal audit group had not issued any recent reports on the forest management 
program. However, to comply with the Crown Forest Sustainability Act and the terms and 
conditions of an assessment under the Environmental Assessment Act, the Ministry is required 
to carry out an audit of each forest management unit every five years. At the time of our 
fieldwork, audits, which included the findings of registered professional foresters, had been 
completed for 20 forestry units. Where appropriate, we relied on those audits to reduce the 
extent of our testing. 


OVERALL AUDIT CONCLUSIONS 


The Ministry does not have sufficient information to adequately meet its obligation to report 
annually on the management of Ontario’s Crown forests. In addition, the Ministry has = yet 
completed the transition from directly managing many aspects of forestry to eae caaiate oe 
appropriate oversight and monitoring procedures to ensure that forestry companies comply 
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legislation and ministry policy and to ensure that ministry resources are managed with due 
regard for economy and efficiency. Specifically, we noted the following: 


yee 


Although the Ministry is required by the Environmental Assessment Board to report 
annually on the management of Ontario’s Crown forests, the last report issued was for the 
1995/96 fiscal year. 


In 25% of the forest management units audited by registered professional foresters, due to 
a lack of information, the foresters could not assess the harvest area successfully 
renewed, which is a key measure of forest sustainability for the Ministry. 


The forest resources inventories, which are detailed descriptions of forested areas, were 
over 20 years old in 16% of forest management units. The Ministry cannot readily assess 
the long-term health of the province’s forests unless all inventories are periodically updated. 
Nevertheless, this was a substantial improvement over our last audit of the program in 1994 
when 55% of the units had outdated inventories. 


Although the Ministry received, thoroughly reviewed and approved documentation 
submitted by forest management companies, such as forest management plans and work 
schedules, it needed to require forest management companies to provide a more detailed 
analysis of major variations from planned results, including recommendations for corrective 
action. 


Due to the lack of markets for some species, forest resources processing facilities often did 
not accept all the wood that they had planned to consume, which hindered plans for forest 
sustainability. Consequently, overall forest management planning needed to incorporate 
more in-depth analysis of the economic factors affecting the industry. 


Independent foresters were generally satisfied with the renewal and maintenance work that 
had been carried out in the majority of the management units they audited. However, the 
Ministry needed to implement corrective action in the remaining units to ensure that 
renewal and maintenance work achieved planned levels. 


In areas where the Ministry continued to perform compliance inspections after the 
responsibility for such inspections had been delegated to forest management companies, 
ministry inspectors found significantly more violations than industry inspectors did. This 
indicated a need to upgrade the forest industry inspection process and develop a more 
formal ministry oversight program. 


District offices were inconsistently imposing penalties for non-compliance and stricter 
penalties were likely warranted where warnings and less severe measures were not having 
the desired deterrent effect. 


Since our 1998 financial controls review, the Ministry has significantly reduced the number 
of overdue stumpage charges receivable from forest management companies. However, to 
ensure that forest management companies were submitting all stumpage charges due based 
on timber actually harvested, the Ministry needed to take more effective action to correct 
the widespread problems noted during its stumpage audits of the companies. 
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DETAILED AUDIT OBSERVATIONS 


MEASURING AND REPORTING ON FOREST 
SUSTAINABILITY 


REPORTING REQUIREMENTS 


The Crown Forest Sustainability Act defines sustainability as long-term Crown forest health 
consistent with the principles of a large, healthy, diverse and productive forest that conserves 
ecological processes and biological diversity. Sustainability must balance the economic demands 
of the production of wood and wood products with the provision of proper environmental 
conditions for wildlife, protection against floods and soil erosion, maintenance of recreational 
uses and protection of the water supply. The Act outlines good forestry practices to ensure the 
proper implementation of appropriate harvest, renewal and maintenance activities. 


Under the Crown Forest Sustainability Act, the Minister is required to prepare and submit to 
the legislature a report on the state of Crown forests at least once every five years. In addition, 
under the terms of an assessment mandated under the Environmental Assessment Act, the 
Ministry is required to prepare an annual report on forest management. The Ministry is bound 
by the terms of this assessment until the year 2003, at which time the Minister of the 
Environment will decide whether to extend, amend or require a new environmental assessment. 


This annual report on forest management must contain a number of statistics, including a 
summary of Ontario’s forest base, harvest volumes, spending on renewal activities, and a 
summary of renewal effectiveness, monitoring results and forestry revenues. Such a report 
would provide the Legislature, the Ministry and the public with timely information with which to 
assess the management of Crown forests and, where necessary, highlight matters requiring 
corrective action. However, the Ministry has not prepared the required report since the 1995/96 
fiscal year. 





Recommendation © 


The Ministry should report annually on the management of Crown forests, as 
required under the Environmental Assessment Act, so that any necessary 
corrective action can be taken on a timely basis. 


Ministry Response 


The Ministry will renew its efforts to produce these reports. The 1996/97 and 
1997/98 provincial annual reports on forest management are scheduled for 
completion by October 2000. The 1998/99 provincial annual report on forest 
management is scheduled for completion by December 2000. Subsequent 
provincial annual reports on forest management will be scheduled for e 
completion within 18 months following the fiscal year being reported on. The 
1995-2000 state of the forest report is on schedule for completion by 


December 2001. 
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MEASURES OF EFFECTIVENESS 


Each year, the Ministry develops a number of key performance measures that are reported 
through the government business planning process. In its 1999/2000 Business Plan, the Ministry 
proposed a measure of forest sustainability termed harvest area successfully renewed. Forest 
renewal can be achieved naturally (natural seeding) or artificially (direct seeding or planting). 
To be deemed successfully renewed, a forest stand must require minimum stocking, be 
composed of the desired species, reach a minimum height and be free of competing vegetation 
that may impede growth. Forest assessments are conducted by forest management companies 
to determine the status of the forest condition, the effectiveness of renewal treatments, and the 
need for and type of remedial action required if an area is not successfully renewed. 


The Ministry’s Business Plan noted that measurement of the harvest area successfully 
renewed would be based on the results of a sample of independent forestry audits performed 
during the year. The Ministry made a commitment to achieve a 96% success rate for renewals 
and report on any major variations in renewal success. 


The Ministry reviewed the independent audits of 20 forestry management units that had been 
completed in 1998 and 1999 and noted that 5 of the 20 units had not reported on harvest area 
successfully renewed. The results for those units had not been reported because insufficient 
information was available, specific areas had not been tracked over the years or records had 
not been kept on renewal assessments. The results for the remaining units ranged from 53% to 
100% renewal success in the areas surveyed. However, several reviews included qualified 
conclusions, noting, for example, that only a small portion of the management unit area had 
been surveyed. 


The Ministry plans to accumulate the results of this measure each year so that, after five years, 
the sample will be large enough to draw overall conclusions on the state of renewal of the 
Crown forests in Ontario. The Ministry intends to use aggregate results to monitor provincial 
trends and identify areas requiring corrective action. However, any conclusion on the state of 
the renewal of Crown forests in Ontario that is based on such incomplete information may be 
misleading. 








ee 


Recommendation 


To ensure that sound conclusions can be drawn on the state of renewal of 
Ontario’s Crown forests, the Ministry should require that the necessary forest 
assessments be carried out in each forest management unit. 


Ministry Response 





The Ministry will take steps to ensure forest managers comply with their 
reporting obligations for renewal activities. The Ministry will also enhance the 
assessment of these results to draw sound conclusions on the state of 
renewal of Ontario’s Crown forests. 
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FOREST MANAGEMENT OPERATIONS 


As of December 1999, 45 of the 68 forest management units had been licensed and another ‘3 
units had been assigned the responsibilities of a licensee under prior agreements. The Ministry 
plans to issue licences to the 13 deemed licensees by the end of the 2000/01 fiscal year. The 
management units assigned to licensees and deemed licensees accounted for over 95% of 
forestry activity over the past five years. The Ministry manages the 10 remaining units directly. 


FORESTRY INFORMATION 


Forest resources inventories classify forests and other areas into broad physical categories, 
such as productive forests, non-productive forests, non-forested land and bodies of water. Such 
inventories provide a baseline of information for assessing the health of Ontario forests, 
developing management plans and monitoring the activities of forest management companies. 
Traditionally, the Ministry’s objective was to update its inventory for each forest management 
unit at least every 20 years. In 1994, the Environmental Assessment Board made it mandatory 
to update and make publicly available forest inventory data for each forest management unit. 


The 1994 Crown Forest Sustainability Act requires the Ministry to produce a forest 
information manual clearly stating current policies and requirements for forest resources 
inventories as well as responsibilities for inventory preparation. However, the Ministry had not 
yet completed this manual to clarify policies, requirements and responsibilities for forest 
resources inventories. 


In our 1994 audit of this program, we reported that approximately 55% of the existing forestry 
resources inventories were over 20 years old and required updating. We reviewed the 
Ministry’s current forest resources inventories and, while significant progress had been made, in 
11 management units, or over 16% of the units, forest resources inventories were over 20 years 
old and required updating. The Ministry plans to have all inventories updated by the end of the 
2001/02 fiscal year. We also noted that the roles and responsibilities of the forest industry in 
updating these inventories would need to be clarified in the forest information manual. 


Up-to-date information is necessary to ensure that the forestry planning process is reliable. For 
example, an independent audit of a forest management unit by a registered professional forester 
indicated that the difference between planned and actual harvest was predominantly the result 
of differences between the inventory description of the forest and the actual conditions that 
existed on the ground. In addition, ministry staff in the district offices we visited indicated that 
outdated forest inventories seriously hampered their planning efforts. Furthermore, two of the 
districts we visited questioned the accuracy of newly completed inventories. Current, accurate 
information is essential for forestry planning and monitoring. 














Recommendation 


To comply with legislation and the 1994 requirements of the Environmental 
Assessment Board and to assist in forestry planning and monitoring, the 
Ministry should: 

* produce the required forest information manual to establish policies 
regarding the collection of forest resources inventories; and 
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* implement adequate oversight procedures to ensure that forest | 
inventory information is accurate, publicly available and up-to-date for 
each forest management unit. 





Ministry Response 


The Ministry is pleased to have the Provincial Auditor recognize the 
significant progress made in updating the forest resource inventory. The 
Forest Information Manual will also describe the roles and responsibilities of 
the Ministry and the forest industry in ensuring the information remains 
current in the future. 


The Ministry will complete the stakeholder consultation on the Forest 
Information Manual, and will bring forward the Manual for regulation under 
the Crown Forest Sustainability Act in fiscal 2000/01. The Ministry will also 
undertake a review of the delivery of the forest resources inventory, including 
a quality assurance component, to identify and act upon cost-effective 
improvements that will meet the needs of forest managers and the public. 











PLANNING AND HARVESTING 


Under the Crown Forest Sustainability Act, forest management plans are to be prepared 
every five years for each forest management unit. Forest management plans are to contain: an 
economic profile of the unit, including a forecast of available wood supplies; estimates of 
harvest volume and sales demand; and a determination of the economic, social and 
environmental impact of forestry operations. These plans are designed to ensure that the unit is 
managed on a sustainable basis. The Act also requires annual work schedules that detail the 
areas to be harvested and the species and volume of trees to be cut. These schedules must be 
consistent with the forest management plan. 


Forest management planning must balance a number of diverse factors to ensure sustainability. 
For example, over-harvesting has a negative affect on sustainability as renewal should keep 
pace with harvesting to optimize the resources available. If harvesting of selected species is 
practised, the makeup of the forest, or its biodiversity, may change, allowing less desirable tree 
species to dominate. If the forest is not cut as planned, the wood may Over-mature and lose 
much of its merchandisable volume. 


We examined the plans and work schedules prepared by forest management companies in 
seven district offices. All of the required plans had been submitted and were completed in a 
format consistent with the legislative requirements to ensure sustainability. They had been 
reviewed in detail and approved by ministry staff. All work schedules had been reviewed for 
consistency with the forest management plans and properly approved by ministry staff. 
However, over half the district offices reported that forest management companies had 
continually and significantly over- or under-harvested. 


While there may be valid reasons to over- or under-harvest from one year to the next, the 
actual harvest over the last six years for each of the 68 Management units ranged from 20% to 
122% of the planned levels. The vast majority of units were under-harvested, with the primary 
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reason being a lack of markets for the planned wood supplies. Province-wide, the actual 
harvest was 72% of planned levels, as noted below: . 


Province-wide Planned versus Actual Harvest Volume 





35 4 
| &] Planned Harvest | 


&§ Actual Harvest 





Millions of Cubic Metres 


























1993/94 1994/95 1995/96 1996/97 1997/98 1998/99 


Source: Ministry of Natural Resources data 


To assist in monitoring forest operations, the Ministry requires forest management companies to 
submit summary reports comparing planned to actual harvesting. These reports are to include 
an analysis of any variance from planned activity as well as specific conclusions and 
recommendations for improvement when variances are significant. 


We reviewed these summary reports in the seven districts we visited and noted a need for 
more complete reporting of variations from planned harvest levels. General explanations were 
usually provided for differences between planned and actual results, but thorough quantitative 
analyses of the differences and recommendations for improvement were not normally included. 
More detailed explanations could highlight the significance of each factor and identify areas 


requiring corrective action. 














Recommendation 
| aged on a sustainable 


To help ensure that forest management units are man 
ance with approved 


basis and harvest operations are carried out in accord 

plans, the Ministry should require: 

* athorough quantitative analysis of all substantial variations between 
planned and actual harvests; and 


* specific conclusions and recommendations for corrective action 
whenever there are significant variations from planned harvest volumes. 
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Ministry Response : 
The Ministry will take steps to ensure sustainable forest licensees provide a 


quantitative analysis of substantial variations between planned and actual 
harvest levels, and will take corrective actions to address these variances. 


The Ministry recognizes that actual harvest levels have been lower than 
planned. The Ministry has turned this situation into an opportunity, in that it 
has positioned itself to address: 


* the expansion of Ontario’s parks and protected areas by 2.4 million 
hectares, under Ontario’s Living Legacy initiative, resulting in reduced 
areas available for timber harvesting; 


¢ the projected decline in the Crown conifer wood supply, resulting 
naturally from an aging forest; and 


¢ the surplus of underutilized Crown hardwood species, which are currently 
being offered to businesses that will create new economic development 
and employment opportunities. 


While these factors will reduce the variance between planned and actual 
harvest levels in the future, planning, operational and economic uncertainties 
in the forest products industry will continue to contribute to the variances 
noted in this audit. 











FOREST RESOURCE PROCESSING FACILITIES 


The Crown Forest Sustainability Act states that a person shall not operate, construct or 
modify a forest resources processing facility except in accordance with a licence issued by the 
Minister. The Minister may issue a forest resource processing facility licence to a person if the 
Minister is satisfied that the person has a sufficient supply of forest resources to operate the 
facility. In Ontario there are approximately 120 licensed facilities, including saw mills and pulp 
and paper mills. Many forest management companies have integrated operations and run their 
own mills. Licences are granted for five years. The holder of a licence is required to report 
annually to the Minister detailing information on production, employment and sales. 


The licensing and annual reporting process is intended to allow the Ministry to evaluate whether 
a sufficient supply of resources is available to operate a facility and whether sufficient facilities 
are available to process the planned harvest. However, during our audit, we noted that 13 forest 
resources processing facilities were operating without a current licence. We also noted that 
annual reports were not received by the Ministry on a timely basis and in some cases did not 
include all the information required. 


The lack of markets for some species was a major factor in many mills failing to meet 
commitments they had made to forest management companies to accept forest resources. The 
Ministry did not monitor the extent to which mills failed to meet commitments to forestry 
companies or whether commitments corresponded to expected harvests. For example, one 
forest management plan outlined commitments from mills to receive forest resources that 
represented less than 25% of the expected harvest. In general, the forest management plans 
we reviewed had not adequately assessed future market demand or the forest resource 
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requirements of the mills. Planning for long-term forest sustainability is not practical without a 


reasonable expectation that mill commitments will be honored and forest resources can be 
processed. 


Recommendation 


To promote sustainable forest management practices and optimize economic 
opportunities, the Ministry should ensure that: 


* all operating mills are licensed, as required by the Crown Forest 
oo Sustainability Act; 
¢ the required annual information reports from mills are complete and 
__ received on a timely basis; and 
¢ forest management company plans consider market demand as well as the 
_ capacities and requirements of the local mills. 


Ministry Response 
The Ministry will take steps to ensure all operating mills are licensed and 
information reports are received. The Ministry will also ensure that forest 


management plans adequately assess market demand and the forest 
resource requirements of local mills. 


FOREST RENEWALAND MAINTENANCE 


Renewal and maintenance activities are designed to ensure the proper renewal of the forest 
after harvest or natural depletions caused by fire or insect infestation. Renewal and 
maintenance include activities such as site preparation, seeding, planting and insecticide 
application. 


Forest management companies are required to include in their forest management plans 
documentation of the desired outcome of renewal and maintenance activities, the planned 
method of reaching that outcome and the standards that will be used to assess success. These 
activities are to conform to the renewal specifications, standards and other instructions that are 
specifically designed for each management unit by a registered professional forester. 


We reviewed a sample of independent forestry reports completed in 1999 and representing all 
three regions of the province. These reports assessed forestry operations over the preceding 
five years, highlighted problems and included recommendations for improving forest renewal 
and maintenance operations. We noted that, for the majority of the management units they 
audited, the independent foresters were generally satisfied with the renewal and maintenance 
work that had been carried out, and they reported that the work performed was consistent with 
the renewal plans designed for these management units. 

However, for the remaining management units, the foresters were generally not satisfied with 
the renewal and maintenance work that had been carried out over the period of their audit. The 
foresters commented that, in these units, the lack of renewal and maintenance ae en 
primarily due to insufficient funding. For example, in one management unit, less than 60% of the 
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area that had been assessed for natural renewal was regenerating as planned. The poor results 
were attributed to a lack of funding to perform the necessary work to clear the area of 
competing vegetation. 


On a province-wide level, there has been a significant increase in the total area of annual 
renewal of Crown land. In addition, as can be noted from the table below, in recent years, 
renewal has approached harvest levels: 


Forest Renewal versus Harvest 


250 - 
Harvest 


P Renewal 


200 


150 + 


Hectares (000s) 


100 + 


50 + 
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Source: Ministry of Natural Resources data 


Based on our visits to district offices and a review of forest management company files, we 
noted a general lack of overall analysis and explanation of variances between actual renewal 
rates and renewal targets. In those units where renewal efforts were significantly below 
planned levels, the corrective action that needed to be taken had often not been identified. 








Recommendation 





To ensure that all forestry companies manage their units on a sustainable 
basis and achieve sufficient renewal success, the Ministry should: 





* require companies to identify, explain and report on major variations 
between actual and planned renewal and maintenance activities; and 
* identify units where the renewal and maintenance work consistently falls 


short of target levels and implement corrective action to help achieve 
renewal success. 
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‘Ministry Response 





_ The Ministry is pleased to have the Provincial Auditor note the Significant 
increase in the total area of annual renewal of Crown land. The Ministry will 
take steps to ensure sustainable forest licensees comply with their forest 
renewal and maintenance obligations as described in their licences; address 
_the renewal and maintenance variances noted; and take corrective action 
where necessary, to ensure renewal success. 








COMPLIANCE MONITORING 
COMPLIANCE INSPECTIONS 


Each licensed forest management company is required to prepare a forest compliance plan to 
monitor, inspect and report on its forest operations. The plan is designed to ensure compliance 
with legislation, forestry plans and ministry guidelines and requires the approval of the Ministry. 
As of April 1, 1998, the lead role for compliance inspections was delegated to forest 
management companies under the terms of the sustainable forest licence. Previously, the 
Ministry performed these inspections. 


The Ministry’s role is to monitor the inspection process of licensed forest management 
companies to ensure that they carry out their responsibilities as required. However, over a 
three-month period in 1999, the Ministry performed over 650 inspections in areas where the 
responsibility had been transferred to forest management companies. In many of these areas, 
forest management companies had implemented their own compliance plans and inspection 
cycles. 


The results of each inspection were summarized for input into the Ministry’s compliance system 
in terms of “in compliance” (no problems were noted), “in compliance with comments” (minor 
issues were noted) and “not in compliance” (a legislative transgression occurred). As can be 
seen from the chart below, in areas where both the Ministry and the forest management 
companies performed inspections, ministry inspectors generally found significantly more 
instances of non-compliance. 
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Ministry versus Industry Inspections 





Industry 
Pj Ministry 











Percent 

















In Compliance In Compliance with Comments Not in Compliance 


Inspection Results 


Source: Ministry of Natural Resources data 


We reviewed a number of company compliance plans and found that the majority of the 
inspection strategies were general and not based on the specific risks associated with the 
management unit. Also, our analysis of violations over the 1997/98 and 1998/99 fiscal years 
revealed that half the offences for unlawful harvesting and harvesting without a licence 
occurred in four districts. Since the results of most ministry and company inspections are 
recorded in the Ministry’s compliance system, this system could be used to help develop a risk- 
based compliance strategy for the Ministry and forest management companies. 


The Ministry spent $5.2 million during the 1998/99 fiscal year to employ over 40 staff to 
perform monitoring and inspection functions. Although this continued inspection program often 
duplicated the work of company inspectors, the primary goal of the inspection process was to 
ensure compliance, and the situation at the time of our audit indicated a need for a continued 
ministry presence. Alternatives need to be considered, such as more directly overseeing 
company inspectors where necessary or performing ministry inspections on a cost-recovery 
basis. 








Recommendation 


To help ensure compliance with the Crown Forest Sustainability Act through 
an effective and economical inspection process, the Ministry should: 


¢ identify areas at high risk of non-compliance where ney inspection 
staff should focus their efforts; 
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* provide forest management companies with information based on 
: previous ministry experience and analysis of trends in violations and non- 
compliance to help them improve their forest compliance plans; 
* where necessary, monitor and upgrade the skills of forest management 
company inspectors; and 
* assess the inspection process as it is currently operating and resolve any 
_ concerns that have resulted from the transfer of inspection 
responsibilities to the industry. 




















Ministry Response 


The Provincial Auditor’s findings support the work of the Ministry in 
developing the compliance reporting system that is just beginning to produce 
trends and records of forest operations. These records, combined with 
independent forest audits, will over time support an effective, economical 
inspection process. The Ministry agrees with the Provincial Auditor’s findings 
_and will initiate a review of its compliance and enforcement activities to 
examine and act upon these audit recommendations. 





ENFORCEMENT 


To protect Ontario’s forests from activities that may impair sustainability, the Crown Forest 
Sustainability Act outlines a number of penalties that can be imposed for violations such as 
illegal harvesting, wasteful practices and failure to comply with an order from the Ministry. The 
Ministry can: issue a warning; issue a repair, stop-work or compliance order; or impose an 
administrative penalty of up to $15,000 and a stumpage penalty of up to five times the value of 
the forest resources harvested in contravention of the Act. In the most severe cases, the 
Ministry can suspend or cancel a licence and the courts can impose fines of up to $1 million. 


The Ministry’s enforcement policy encourages district offices to utilize the full range of 
enforcement provisions in a progressively incremental manner depending on the frequency, 
severity and significance of the violation. All violations are to be recorded in the Ministry’s 
computerized enforcement system. During the first nine months of the 1999/2000 fiscal year, 
although information from all districts had not yet been input, the system recorded over 450 
violations. 


We reviewed the Ministry’s enforcement process and identified the following weaknesses: 


¢ The enforcement system did not identify or track repeat offenders. For example, during our 
district visits, we noted that a forest management company had committed a number of 
serious violations over the last four years. These violations included unauthorized 
harvesting, wasteful practices and failure to comply with a compliance order. The company 
had been assessed administrative penalties five times, received three warnings, three 
compliance orders and a voluntary compliance request. The penalties were neither based 
ona complete history of the company’s violations nor imposed in a progressively ~ 
incremental manner. A readily accessible history of each offender would assist ministry 


staff in imposing penalties appropriately. 
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* The Ministry used penalties and warnings inconsistently among districts. In some districts, 
warnings were repeatedly issued for the same infraction when a more severe penalty might 
have been more appropriate. Our survey of district offices indicated that some ministry 
staff felt that warnings were only effective in certain circumstances while others felt that 
warnings, if used properly, were an effective enforcement tool. 


* Escalating penalties were often not imposed where appropriate. For example, wasteful 
practices were often dealt with by issuing a warning or request to voluntarily clean up 
timber. The Ministry’s rationale was that the wasteful practices usually involved small 
quantities of timber and were generally not a significant violation. However, we reviewed 
the nine-month period from April to December 1999 and found that unrestrained wasteful 
practices can accumulate into a significant problem. For example, two districts had over 
100 instances of wasteful practices. The vast majority of these incidents resulted in a 
warning to the forestry company. In these two districts, warnings did not have the desired 
deterrent effect and more severe measures were likely warranted. 














Recommendation 


To improve overall compliance with the Crown Forest Sustainability Act and 
protect Ontario’s forests from activities that could impair forest sustainability, 
the Ministry should: 


¢ implement procedures to ensure that the information recorded in the 
compliance system is complete and can be used for management decision- 
making purposes such as identifying and tracking repeat offenders; 


* review the enforcement approaches used by district offices to ensure that 
penalties are imposed in a consistent manner; and 


* analyze the extent of specific violations to identify trends and areas where 
more severe penalties or other corrective action may be required. 


Ministry Response 


The Ministry agrees with the Provincial Auditor’s findings. The compliance 
reporting system will produce trends and records of forest operations to 
support decision making related to enforcement. The Ministry will initiate a 
review of its enforcement activities and act upon these audit 
recommendations. 








FORESTRY TRUSTS 


To provide funding for forest renewal, the Crown Forest Sustainability Act established two 
trusts: the Forest Renewal Trust for renewal activities after harvesting and the Forestry Futures 
Trust to provide for renewal due to depletion from other causes such as fire, insect infestations 
and forest management company insolvency. The trusts are assets of the Crown but are 
administered by a trust company and audited annually by a public accounting firm. All licensed 
forest management companies that harvest Crown timber are required to pay renewal charges 
into the trusts based on a prescribed rate per cubic metre of timber harvested. 
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Trust Fund Activity and Balances for the Year Ended March 31, 1999 
($ millions) 


Forest Renewal Forestry 
Trust Futures Trust 


115.7 | 11.4 









Funds Beginning of the Year 
Forestry Charges/Other Funding 
Silviculture/Other Expenditures 
Funds Available For Silviculture 

























Source: Trust Funds’ Audited Financial Statements 


The trust company makes payments out of the Forest Renewal Trust based on renewal 
expense information received from licensed forest management companies. Annually, each 
licensed company is required to submit a final invoice report on the renewal activities for the 
preceding year. All renewal receipts and disbursements are accounted for by forest 
management unit and recorded in the name of the licensee. Licensees are required to maintain 
a minimum account balance for each forest management unit. A committee set up under the 
Act authorizes disbursements from the Forestry Futures Trust. In addition to disbursements for 
renewal, payments are also made from the Forestry Futures Trust for other authorized purposes 
such as performing forest resources inventories and independent forestry audits. 


We reviewed the administration of the forestry trusts and noted the following problems: 


¢ As at March 31, 1999, contrary to the trust agreement for the Forest Renewal Trust, 20% 
of forest management companies’ accounts contained less than the minimum required 
balance, and one company was in overdraft. 


¢ The trusts’ accountant reported an arrangement whereby all forestry charges deposited to 
the credit of the Forest Renewal Trust are transferred to the Forestry Futures Trust after 
one to two months. As at March 31, 1999, $3.3 million had not yet been transferred to the 
Forestry Futures Trust. Consequently, over the preceding fiscal year, the accountant 
estimated that the Forestry Futures Trust lost $90,000 to $120,000 of investment income. 
The Forest Renewal Trust benefited from this arrangement. 


¢ Contrary to the trust agreement, final invoice reports, which detail the renewal activities for 
the preceding year, were not consistently submitted to the trustee prior to the final payment. 















Recommendation 
To ensure that the Forest Renewal and Forestry F utures trusts are 
administered in accordance with trust agreements, policy and legislation, the 
Ministry should: 

* monitor Forest Renewal Trust account balances to ensure tha 
management company maintains the minimum balance required so tha 
funds will be available for renewal activities when needed; 

° ensure that forestry charges are transferred to the Forestry Futures Trust 


t each forest 











on a timely basis; and J 
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* ensure that payments from the Forest Renewal Trust are , made only after | 
the required final invoice report is received. 


Ministry Response 


The Ministry will ensure that the minimum balance requirements for each 
sustainable forest licence area account are met by March 31, 2001 and 
continue to be met every March 31 thereafter. 


The Ministry and the trustee have also taken steps to ensure Crown charges 
are properly transferred to the Forestry Futures Trust and that final payments 
from the Forest Renewal Trust are made only after the required final invoice 
report is received. These audit findings will continue to be reviewed in annual 
financial audits of the trusts. 














FORESTRY REVENUE 


Under the Crown Forest Sustainability Act, the Minister, on behalf of the province, collects 
stumpage charges from forest management companies for cut and measured timber. The 
charges are based on harvest volume and tree species and are deposited in the province’s 
Consolidated Revenue Fund. Stumpage charges for the 1999/2000 fiscal year amounted to 
$155.7 million. 


In 1998, we reviewed ministry financial controls and reported that large overdue accounts 
receivable were a problem and that it often took six months after timber was harvested for the 
Ministry to issue an invoice for the applicable stumpage fees. In January 2000, the Ministry’s 
aged trial balance showed that 93% of the balances were under 30 days, and our testing 
indicated that the average time between measuring timber and invoicing had improved to 3.4 
months. The Ministry has made substantial improvements in its administration of stumpage 
charges receivable. 


The Ministry has a program for conducting audits to provide assurance that the province’s 
financial and other interests are adequately protected with respect to the measurement of 
Crown timber and the related stumpage charges. Forestry management companies are audited 
at least once every five years. 


Some observations reported by ministry audit staff for the 1999/2000 fiscal year included: 


* Poor controls over companies’ bills of lading systems resulted in loads of wood being 
undetected and a corresponding loss in revenue for the province. 


* Amunistry district office’s lack of timely reconciliations of a company’s bills of lading made 
it impossible to determine whether the company had met all of its obligations. 


* The company had not utilized the Ministry’s mechanisms to differentiate private wood from 
Crown wood. Consequently, there was no guarantee that wood identified as private, which 
does not require a stumpage charge, was indeed private wood. 


These observations were also described as problem areas in a summary report of the audits 
performed by ministry staff during the 1997/98 fiscal year. This summary report indicated that 
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in many cases the Ministry’s interests had not been adequately protected and, in almost half the 
audits conducted, the proper amount of stumpage charges had not been paid. 





Recommendation 


To ensure that the province receives the proper amounts of revenue, the 
Ministry should systematically address any recurring problems noted through 
its audit examinations of stumpage charges. 


Ministry Response 


The Ministry is pleased to have the Provincial Auditor recognize the 
substantial improvements made in the administration of stumpage charges 
receivable. With a view to continued improvement, the Ministry will continue 
with its practice of undertaking wood measurement audits on 15 to 20 
companies each year, and will systematically address any recurring problems 
noted in these audits. It should be noted that the margin of error in assessing 
Crown charges has been less that 2% in recent years. 
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Ontario Native Affairs 
Secretariat 





BACKGROUND 


The Ontario Native Affairs Secretariat is headed by the Assistant Deputy Attorney General 
and Secretary for Native Affairs and supports the Attorney General and Minister Responsible 
for Native Affairs. The Secretariat employs approximately 60 staff, with a head office in 
Toronto and regional offices in Thunder Bay and Pembroke. 


The Secretariat works with First Nations, Aboriginal organizations and businesses to build 
strong, prosperous and self-reliant Aboriginal communities. The Secretariat also represents 
Ontario in self-government negotiations between Canada and First Nations as required. Its core 
businesses are: negotiations, Aboriginal economic development, corporate coordination of 
Aboriginal affairs and internal business support. Under these businesses, key activities include: 


* conducting land claim negotiations on behalf of the province and implementing land claim 
settlements; 


¢ funding capital projects that are delivered by other ministries; 
* providing core funding for eligible Aboriginal organizations; and 
* promoting Aboriginal economic development. 


The Secretariat’s direct expenditures of $18.7 million for the 1999/2000 fiscal year are detailed 
in the following graph: 
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Ontario Native Affairs Secretariat 
1999/2000 Expenditures 
($ millions) 


Building Aboriginal 
Economies Strategy 
(1.7) 


Core Funding of 
Aboriginal Organizations 
(0.9) ~~ Bip 
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Capital Transfers 
(4.6) 


Transfers for tripartite, 
self-government and 
= constitutional 
negotiations 

(0.6) 


Transfers for land claims 
support 
(4.5) 


Source: Ontario Native Affairs Secretariat 


The Secretariat also works with other ministries to provide cross-government coordination of 
Aboriginal affairs and provides advice to the government on Aboriginal matters. For instance, 
the Building Aboriginal Economies Strategy, which promotes economic growth and employment 
in Aboriginal communities, provides a framework for over 35 programs and services across 11 
ministries. As part of the Strategy, the Working Partnerships Program encourages Aboriginal 
partnerships with the private sector. 


Provincial programs and services delivered to Aboriginal residents are funded and delivered by 
over 16 ministries, with total expenditures exceeding $370 million for the 1998/99 fiscal year. 
These expenditures were for such activities as First Nation welfare allowances and benefits, 
Aboriginal childcare, and health insurance and hospital services. 


AUDIT OBJECTIVES AND SCOPE 


Our audit objectives were to assess whether the Secretariat’s management practices 


adequately ensured that: 


* — internal policies and procedures regarding land claims were being complied with; 


* secretariat programs and services demonstrated value for money over the long term, based 


on business goals, results and accountability; and 

* coordination of Aboriginal affairs across the government met secretariat business plan 
commitments and that ministries received timely and useful information, guidance and 
feedback regarding their Aboriginal-specific programs. 
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The scope of our audit included a review and analysis of relevant secretariat files and 
administrative procedures as well as interviews with appropriate staff at the Secretariat and a 
number of ministries. We also reviewed a number of files relating to capital programs that were 
being administered by other ministries. 


Prior to the commencement of our audit, we identified the audit criteria that would be used to 
address our audit objectives and reviewed these with senior secretariat management. We did 
not rely on the work of internal audit because it had not issued any recent reports on the 
Secretariat. 


Our audit work covered the period to March 31, 2000. Our audit was performed in accordance 
with the standards for assurance engagements, encompassing value for money and compliance, 
established by the Canadian Institute of Chartered Accountants and accordingly included such 
tests and other procedures as we considered necessary in the circumstances. 


OVERALL AUDIT CONCLUSIONS 


We concluded that the Secretariat was generally complying with internal policies and 
procedures regarding land claims. 


The Secretariat’s internal operations were administered with due regard for economy and 
efficiency. However, we concluded that several aspects of the Secretariat’s programs and 
services could not demonstrate value for money. Specifically: 


* While land claim settlements were adequately supported by independent evaluations or 
other appropriate evidence, the Secretariat needed to better ensure the timeliness of 
reporting and accountability by First Nations for the use of secretariat land claim funding 
provided to them for their negotiating costs. 


* For the two secretariat-funded capital programs delivered by two other ministries, which 
accounted for 45% of total secretariat expenditures, we found that secretariat monitoring 
was not supported by standardized formal reporting of sufficient detail and that the 
arrangement whereby the Secretariat funded but ministries delivered these programs added 
no discernible value and blurred the lines of accountability. 


* In providing core funding to Aboriginal organizations, the Secretariat needed to better 
ensure that funding reflected changes in local needs over time and that audit management 
letters on recipients’ operations were obtained and promptly addressed. 


We concluded that the Secretariat’s coordination of Aboriginal affairs across the government 
met secretariat business plan commitments. However, we were not satisfied that the 
Secretariat’s database on provincial expenditures on programs and services delivered to 
Aboriginal people, which were over $370 million for the 1998/99 fiscal year, was timely, 
accessible and detailed enough to be useful for cross-government program coordination or as a 
resource for ministries delivering Aboriginal-specific programs. 


Finally, while we concluded that the Working Partnerships Program has been effectively 
implemented, the process for reporting on Building Aboriginal Economies results needed 
significant improvement. 
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Overall Secretariat Response 























The Secretariat is pleased that the Provincial Auditor’s report contains 
positive comments about each of its core businesses: land claims, corporate 
coordination of Aboriginal affairs and Aboriginal economic development. 


The Secretariat appreciates the suggestions and recommendations provided 
by the Auditor on ways to improve its operations. A number of improvements 
_have already been made or are underway. 


In the area of land claim negotiations funding, First Nations are increasingly 
using the funding guidelines first issued by the Secretariat in 1998. The 
Secretariat continues to work with funding recipients to enhance the level of 
documentation submitted and to improve the timeliness of their reporting. 


The Secretariat agrees that good data on Aboriginal economic development 
is key to measuring the success of the Building Aboriginal Economies 
Strategy and is continuing to work with partner ministries to monitor access 
by Aboriginal people to programs and services and to improve data collection 
and reporting systems. 


The Secretariat is committed to making further improvements to how it 
delivers on its core businesses in order to move closer to its vision of a future 
in which Aboriginal communities have stronger economies and are more 
self-reliant, and where relations between Aboriginal people and their 
neighbours are prosperous and mutually beneficial. 





DETAILED AUDIT OBSERVATIONS 
LAND CLAIMS 


A land claim is a formal statement to the federal or provincial government in which an Aboriginal 
community asserts that the Crown has not lived up to its commitments or obligations with respect to 
Aboriginal or treaty rights pertaining to land. There is little specific legislation governing land claims. 
Most Aboriginal land claim negotiations involve the federal government, which has primary 
responsibility for their resolution. 


Under the Constitution Act the provinces are assigned legislative authority for Crown lands, private 
property and natural resources within their boundaries. As such, provinces may become involved 
because of the historical events that give rise to such claims and because many claims involve the 
assertion of rights with respect to provincial Crown lands, natural resources and private property. 
Other relevant legislation includes Ontario’s Indian Lands Agreement Confirmation Act and its 


reciprocal federal statute. 
n behalf of Ontario. The Secretariat’s 


The Secretariat participates in land claim negotiations 0 , 
e province’s Aboriginal Policy 


specific approach to land claim negotiation is set out in th 
Framework as well as through several publicly available fact sheets. 
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The land claim process can be protracted and contentious and can involve the interests of many 
parties, including private landowners. The process consists of four phases: pre-negotiations, which 
includes claim submission and evaluation; negotiations; settlement; and implementation. All four 
phases include elements of public involvement. The timeframes of land claims vary and can exceed 
20 years. In most instances, the Secretariat contributes to the negotiation costs incurred by the 
Aboriginal community through contribution agreements. 


The goal ofa land claim negotiation is to arrive at a mutually agreeable settlement, one that 
balances the concerns of the province, the federal government, the Aboriginal community and 
all affected third parties. Settlements typically deal with such matters as land ownership, 
clarification of boundaries, future land usage and monetary compensations for loss of land use. 


We concluded that the Secretariat was complying with internal policies and procedures 
regarding land claims and that land claim settlements were adequately supported by 
independent evaluations or other appropriate evidence. We further found that relevant activities 
of other jurisdictions were considered during the development of land claim policy and that 
third-party consultations were conducted on a timely basis. 


However, we noted areas where improvements were needed. The Secretariat needed to better 
ensure accountability and timely reporting by First Nations for the use of secretariat land claim 
funding provided to them for their negotiating costs. 


NEGOTIATION FUNDING 


Once a land claim has been formally accepted for negotiation, the Aboriginal community 
annually submits work plans and proposed budgets to the Secretariat for review and approval. 
Work plans set out such things as planned meeting and communication costs, and legal, 
consulting and technical costs associated with the claim that the Aboriginal community expects 
to incur. Following a review, the Secretariat and the Aboriginal community sign a contribution 
agreement. 


Under the contribution agreement, the Secretariat initially provides 75% of the annual funding 
once all outstanding reports from the Aboriginal community related to prior years have been 
received and approved. The remaining 25% is released following the receipt of interim progress 
and financial reports, which are due on October 31. Final progress and financial reports are due 
shortly after year-end, with audited financial statements due by June 30. All unused funds 
identified by these reports must be returned to the Secretariat or deducted from contributions in 
the next fiscal year. 


In order to effectively monitor the use of the Secretariat’s funds and respond quickly to 
changing circumstances, timely reporting of results is essential. However, we noted that the 
reports needed for such monitoring were not being submitted by First Nations on a timely basis. 
For example, we found that while all final reports for ongoing claims were eventually submitted, 
they were, on average, late by more than 100 days. 


More importantly, in the final year of a negotiation process where no funding for future years 
was contemplated, final reports were not received at all. Therefore, the Secretariat could not 
identify and recover any overpayments made. 


We believe a general policy of holding back the final payment until after final reports and 
audited financial statements are submitted would solve this recurring problem. For instance, one 
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possible mechanism would be to pay 75% of the Secretariat’s agreed-upon funding, as under 
the current arrangement, with the remaining 25% held back until receipt of the final reports and 
audited financial statements following year-end. This could eliminate the need for interim 


progress reports and interim financial reports, saving effort for both the Secretariat and 
Aboriginal communities. 


Our examination of land claim files also showed that the reports submitted by Aboriginal 
communities were often not useful and that the Secretariat’s review of them, in some cases, 
was not documented. For example, there was no required format for reporting on the provincial 
funds provided. As a result, many of these reports did not provide sufficient detail to enable a 
comparison of actual expenditures to budget, identify significant variances for follow-up or 
allow the identification of any overpayments. We also found that variances over 25% from 
budget were not being pre-approved, as required by the contribution agreement, or adequately 
explained. 


Secretariat practice is to fund negotiation costs only through the contribution agreement 
process. The final settlement is meant to address the value of the claim itself. The monetary 
component of a final claim settlement, once paid, is final. However, we noted two current cases 
in which estimated negotiation costs formed part of a final settlement and, accordingly, required 
no reporting of actual costs incurred. Specifically: 


¢ In the first case, we found no documented evidence to support a $550,000 payment for 
negotiation costs arising prior to 1993. 


¢ Inthe second case, during the negotiation process, the Secretariat agreed to include 
projected negotiation costs of $58,000 for a three-month period in the final settlement 
agreement. Before the agreement was signed, the Secretariat’s own calculations indicated 
that over $20,000 of this $58,000 had not been spent yet the full $58,000 was still paid out. 
The $20,000 over-reimbursement cannot be recovered. 





Recommendation 

To improve accountability for the funding of Aboriginal land claim negotiation 

costs, the Secretariat should: 

* consider holding back a percentage of each year’s funding until it has 
received the required final reports and audited financial statements for 

_ thatyear, 

¢ establish a standard format for financial reports and audited financial 

statements to facilitate analysis of significant variances from budget and 

identification of ineligible expenditures and funding surpluses; and 

‘¢ fund projected negotiation costs only through the contribution agreement 

process. 


Secretariat Response 


commendation to hold back a percentage of 


tariat will review the re ; 
The Secretariat will review t to ensure that First 


the current year’s funding having regard for the need en 
Nations have sufficient cash flow to participate in negotiations. 


_ Snr 
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The Secretariat reviews all of the reports submitted by First Nations and 
concurs that in some cases documentation of this process and resulting 
decisions and actions can be improved. The Secretariat continues to 
strengthen reporting requirements in the funding guidelines and will be 
developing: 





° a standard format for financial reports; and 


* requirements that audited financial statements contain sufficient detail to 
enable comparison to budgets set out in the funding agreements and 
identification of funding surpluses and expenditures that were not 
budgeted. 


The Secretariat generally concurs with the recommendation to fund 
negotiation costs through the contribution agreement process only and will 
strive to do so in future. However, in certain situations, it may not be possible 
to achieve a final settlement and resolution to a longstanding claim without 
some flexibility in negotiation funding arrangements. 


In both cases cited by the Provincial Auditor, the payments contributed to 
achieving final settlement of the claims. 











LAND CLAIM COSTING 


We found no system in place to track the total costs of individual land claims, which, as 
indicated previously, often take many years to resolve. Costs associated with specific land 
claims include professional services, legal and negotiation costs, third-party consultation costs 
and settlement implementation costs. 


We acknowledge that the issues involved in land claims can be unique, making meaningful 
comparisons between different land claims difficult. However, we believe that a land claim 
costing system would be useful and would enable full public accountability for the management 
and reporting of the land claim process. Such historical data would be useful in future 
negotiations and help the Secretariat in resolving land claims more efficiently and effectively. 





Recommendation 


To assist with the evaluation of future land claims and enhance accountability 
for the cost-effectiveness of the program, the Secretariat should identify and 
track the total costs of each land claim. 


Secretariat Response 


The Secretariat has tracked transfer payments and costs for travel, 
equipment, meetings, research consultants, alternative dispute resolution 
and public consultation for each land claim for the past two fiscal years. 








The Secretariat will examine the feasibility of tracking additional costs. 
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PROGRAMS ADMINISTERED BY OTHER 
MINISTRIES 


During the period of our audit, the Secretariat funded two major capital programs that were 
implemented by other ministries under the Aboriginal Community Capital Infrastructure Fund. 
These were the Canada-Ontario Infrastructure Agreement, which provides for retrofitting and 
service connections within Aboriginal communities and is administered by the Ministry of 
Northern Development and Mines (MNDM) and the Aboriginal Healing and Wellness Strategy, 
administered by the Ministry of Community and Social Services (MCSS). Total expenditures for 
these two programs in the 1998/99 fiscal year were over $8.9 million, or 45% of the 
Secretariat’s expenditures. 


We found that the arrangement whereby the Secretariat funded but another ministry delivered 
these programs added no discernible value and blurred the lines of accountability. In both 
programs, agreements were in place setting out the respective responsibilities of the Secretariat, 
the implementing ministries and other parties. However, we found little documented evidence 
that the reporting requirements of these agreements were being met. The result was that the 
Secretariat, which funds the programs, was insufficiently monitoring program delivery and often 
did not have sufficient documented evidence that funds were expended for the purposes 
intended. 


We believe that accountability would be improved and administration streamlined by making the 
delivering ministries completely responsible for these programs, with legislative funding flowing 
directly to them. These ministries are in the best position to monitor program expenditures and 
to measure and account for program success. Duplication of effort would be avoided, and the 
Secretariat could concentrate its efforts on programs it delivers directly. 


With respect to the Canada-Ontario Infrastructure Agreement: 


¢ We found that the reports MNDM provided to the Secretariat did not satisfy the terms of 
the agreement that sets out their reporting relationship. Specifically, for the year ended 
March 31, 1999, the Secretariat had received only two one-page reports that did not contain 
any information on variances between budgeted and actual expenditures. 


° We further found that the Secretariat had not collected any information as to the actual | 
number of jobs created by the program. Therefore, it had no way of knowing whether its 
public claim that the program would create 3,000 jobs was being met. 


¢ Inreviewing MNDM project files, we found that the Ministry had ensured the retrofitting 
work was properly completed through periodic site visits. However, the Ministry was not 
ensuring that recipients submitted required financial reports on a timely basis, and we saw 
little evidence of systematic review of these reports. Further, there was no requirement to 
provide a separate reporting of retrofit costs in the audited financial statements of the 
community. Accordingly, MNDM had no independent assurance that the calculations of 
final project costs were complete and accurate, and, accordingly, whether any government 


monies should be recovered. 
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For the Aboriginal Healing and Wellness Strategy administered by MCSS, the Secretariat had 
not sufficiently reviewed backup documentation for reasonableness. Secretariat procedures 
were also inadequate to ensure funds paid were reconciled on an annual basis to budgeted 
amounts and capital reports submitted. 











Recommendation 


In order to eliminate unnecessary overlap of monitoring and administrative 
activities, the Secretariat should in future strive to avoid taking on funding 
responsibility for programs that more appropriately could be administered by 
other ministries. 


While the current funding arrangements remain in place, the Secretariat 
needs to improve its monitoring of these programs in order to know whether 
they are functioning as intended or need corrective action. 


Secretariat Response 


The Secretariat was originally allocated the Aboriginal Community Capital 
Infrastructure Fund (ACCIF) in 1991 as the government wished to have such 
funds managed corporately in the Secretariat, rather than disbursing them 
among a number of ministries. Memoranda of Understanding between the 
Secretariat and its partner ministries clearly define respective roles and 
responsibilities. The Secretariat no longer provides capital funding for the 
Aboriginal Healing and Wellness Strategy. 


The Ministry of Northern Development and Mines (MNDM) monitors its 
disbursement of ACCIF grants to First Nations while the Ministry of 
Community and Social Services (MCSS) has monitored its disbursements of 
ACCIF grants to Aboriginal organizations. Both MNDM and MCSS in turn 
report regularly to the Secretariat by means of verbal and written reports. 


The Secretariat will work with MNDM to standardize regular formal reporting, 
the requirements of which will become part of an existing Partnership 
Agreement between the Secretariat and the Ministry. 
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CORE FUNDING FOR ELIGIBLE ABORIGINAL 


ORGANIZATIONS 


Under the Aboriginal Organizations Core Funding Program, the Secretariat provides funding for 
eligible administrative and operating costs for three Aboriginal organizations: the Ontario Native 
Women’s Association, the Chiefs of Ontario and the Ontario Federation of Indian Friendship 
Centres. Each organization submits a budget and work plan for its upcoming year, which, 
following secretariat review and approval, forms the basis for a contribution agreement 
between the two parties. Funding recipients are required to submit interim financial reports at 
the end of October and year-end progress and financial reports 30 days after year-end. Audited 
financial statements are due 90 days after year-end. 


For the 1999/2000 fiscal year, secretariat funding for this program amounted to $877,072. 


We found the Secretariat’s controls over budget requests and approvals and the monitoring of 
program expenditures could be improved. Specifically: 


¢ Funding was historically based. Each organization received the same amount of funding 
from year to year, despite significantly changing expenditure patterns within the 
organizations themselves. For example, for periods where information was available, 
one organization’s overall expenditures increased from $1.2 million to $3.1 million while 
another organization’s overall expenditures decreased from $14.8 million to $5.9 million. 
In our opinion, core funding needs should be assessed in relation to total activity. 


¢ In some cases, management letters on funding recipients’ operations were available and 
should have been forwarded to the Secretariat as part of the terms of the contribution 
agreement, but were not. For these cases, we found no evidence that the Secretariat had 
made any effort to obtain them. In one case, the withheld management letter documented 
significant control problems at the funded organization, including unbudgeted expenditures 
within the core programs the Secretariat had been funding. 





/ Recommendation | 





To improve the budgeting process and the monitoring of expenditures for 
the Aboriginal Organizations Core Funding Program, the Secretariat should 
ensure that: 

° budget submissions are sufficiently detailed to enable the Secretariat to 
provide funding commensurate with the demand for and value of the 
services to be provided; and 

* all management letters are obtained, relevant issues identified and 
significant control deficiencies are followed up on a timely basis. 
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Secretariat Response 


The Secretariat agrees that budget submissions should be sufficiently 
detailed and ensures that they are. 


As there have been cases in which management letters were not provided 
promptly, the Secretariat will highlight this requirement in the schedule of 
required reports appended to the contribution agreements. The Secretariat 
will continue to work to ensure that management letters are submitted with 
audited financial statements and that control deficiencies are addressed 
expeditiously. 


The Secretariat is planning to conduct a review of core funding in 2000/01 and 
will take this recommendation into account. 











COORDINATION OF ABORIGINAL PROGRAMS 
INFORMATION FOR DECISION MAKING 


According to secretariat records, gross provincial expenditures at 16 ministries delivering 
provincial programs to Aboriginal residents were over $370 million for the 1998/99 fiscal year. 
These expenditures were for such activities as First Nations welfare allowances and benefits, 
Aboriginal childcare, and health insurance and hospital services. In order to help fulfil its 
corporate coordination mandate, the Secretariat recognizes the need for and maintains a 
database of information on these programs. It uses the database to support the development of 
corporate Aboriginal policy and to manage its Aboriginal-specific relationships at both federal 
and provincial levels. 


The Secretariat annually sends out a request for database information to each applicable 
ministry. These annual submissions are summarized and consolidated by the Secretariat into an 
Aboriginal Targeted Expenditure Summary. 


At the time of our audit, the Secretariat was requesting only that ministries provide a description 
of each program and basic financial information. The ministries provided no explanations for 
major expenditure variances from year to year. There was also no reporting of program results 
or service information such as changes in demand levels, numbers of clients served, or relevant 
demographic trends or changes. Collecting this type of supplementary information would allow 
the Secretariat to better predict and plan for future Aboriginal-specific resource needs. 


In our opinion, one way to improve the database would be to supplement it with the types of 
output-based program and service information being gathered through the Building Aboriginal 
Economies Strategy and the Working Partnerships Program. This would enhance the 
Secretariat’s role as corporate coordinator and as a proactive policy advisor. 


We also had concerns about access to the database. Although 16 ministries were contributing 
to it, the database was readily accessible only by the Secretariat. In our discussions with 
ministries’ program staff, we were informed that such a database would be useful to them. 
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Recommendation 


To improve the usefulness of its database as a tool for government-wide 
coordination of provincial programs and services with expenditures of over 
$370 million delivered to Aboriginal residents, the Secretariat should: 


¢ obtain information on program services and results in addition to the 
financial information it currently collects; 


* integrate the information obtained on the Building Aboriginal Economies 
Strategy and the Working Partnerships Program into the database; and 


© take steps to make the database directly accessible to program managers 
__ in the 16 ministries delivering programs and services to Aboriginal 
residents. 


Secretariat Response 


_The Aboriginal database was intended to collect basic expenditure 

_ information, not to obtain output-based program and service information. 
The Secretariat will consider expanding the purpose of the database. This 
would require changes to the submission guidelines, plus increased work by 
the Secretariat and other ministries. The Secretariat would need to consider 
the value of this additional information in relation to the increased demand on 
corporate resources that collecting and managing it would require. 


The database is made available directly to senior officials on a regular basis 
and to program staff upon request. The Secretariat acknowledges that 
database information could be more widely distributed to program managers. 


The Secretariat agrees that there is utility in integrating the Aboriginal 
Expenditure Database with Building Aboriginal Economies and Working 
Partnerships performance results and has commenced work on this 
integration. 





BUILDING ABORIGINAL ECONOMIES STRATEGY 


The Building Aboriginal Economies Strategy (the Strategy) provides a framework for over 35 
programs and services across 11 ministries. Its goal is to increase economic capacities and 
opportunities in Aboriginal communities and to promote self-reliance, growth and investment, an 
improved business environment and the creation of jobs. 
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The Secretariat reports on the Strategy to the Deputy Ministers’ Committee on Aboriginal 
Issues. Its role is one of coordination, data gathering and provision of information. The 
Secretariat directly manages the five-year, $11.8 million Working Partnerships Program, 
established as part of the Strategy in 1998 to encourage more Aboriginal partnerships with the 
private sector. Key program components include a Partnership Development Advisors 
Initiative, workshops and forums, partnership development resource kits, an Aboriginal business 
website, recognition awards and scholarships, and an Economic Renewal Initiative. We 
concluded that the Working Partnership Program has been effectively implemented. 


The Secretariat also facilitates and oversees the development and implementation of the 
Strategy’s performance measures. It compiles and analyzes periodic submissions by ministries 
on the results of the programs they deliver to Aboriginal residents. To this end, the Secretariat 
developed six generic performance measures that were intended to be adapted by ministries for 
reporting on their specific programs. 


As of March 31, 2000, the Secretariat has compiled and prepared two reports on results of the 
Strategy from April 1, 1998 to September 30, 1999 based on reports submitted by the ministries. 
We found this process needed significant improvement, as most of the reported results were not 
attributable to the Strategy. Rather, they were often either results from programs in existence 
prior to the Strategy’s launch or not from the period being reported on. For example: 


* Forty-four percent of the jobs reported as created under the Strategy were based on a 
February 1996 ministry report on Aboriginal employment in the forestry industry. 


* One of the achievements listed was the completion of education and training programs by 
7,452 Aboriginal people. However, 87% of those individuals had participated in these 
programs prior to the period being measured. 











Recommendation 







In order to properly assess the performance of the Building Aboriginal 
Economies Strategy, the Secretariat should ensure that reported results 
actually arise from Building Aboriginal Economies Strategy initiatives. 


Secretariat Response 


The Secretariat is aware of the need for good data in order to measure results 
of the Building Aboriginal Economies Strategy and is working with ministries 
to improve their data collection and reporting. In some cases, until ministries 
are able to provide actual data, estimates will continue to be utilized. 














oo eee 


250 Office of the Provincial Auditor 











MINISTRY OF TRANSPORTATION 


Monitoring School 
Purpose Vehicle Safety 





BACKGROUND 


In conjunction with our audit of pupil transportation grants provided to school boards by the 
Ministry of Education, we determined that it was important to consider the Ministry of 
Transportation’s role in ensuring that pupil transportation is safe. 


The responsibility for ensuring that student transportation services are safe is shared among 
several authorities, the respective responsibilities of which are set out in the following table. 


[“Tegisiation | Responsibilities ——SS—id 

civil liability e ensuring safe design of bus routes, safe pick-up and 
drop-off procedures and locations, and safe behaviour 
on school purpose vehicles 

¢ enforcing provisions that deal with safety in contracts 

with bus operators 

¢ enforcing provisions of the HTA and associated 
regulations designed to ensure that school buses are 
mechanically fit 

* ensuring that school purpose vehicles are in compliance 
with the Motor Vehicle Safety Act 

e enforcing driver licensing provisions of the HTA 

assisting police in investigating accidents that involve 

school buses 

developing and maintaining safety standards for school 

purpose vehicles and investigating serious accidents 

enforcing compliance with road safety provisions of the 

HTA and investigating accidents 






















School Boards 














Highway Traffic Act 
(HTA) 


Ministry of 
Transportation 




















Motor Vehicle Safety 
Act 


Highway Traffic Act 


Transport Canada 









Police Forces 


The Ministry’s responsibilities with respect to school buses are carried out by the Carrier Safety 
and Enforcement Branch of the Ministry’s Safety and Regulation Division. The Branch has 65 
staff at the Ministry’s head office in St. Catharines and 325 enforcement personnel at the 
Ministry’s 17 district offices. Enforcement personnel monitor an estimated 68,000 active 


commercial carriers with about 191,000 vehicles. Of these carriers, approximately 1,000 are 


school bus operators with an estimated 16,000 buses—the exact numbers of school bus 
s not have a separate classification for 


operators and buses are not known as the Ministry doe 
them. 
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The Branch’s efforts to ensure that pupil transportation services are safe include inspections of 
school purpose vehicles for mechanical fitness, audits of vehicle maintenance facilities and 
checks to ensure that drivers have the required licences and properly maintain their logs on 
hours of work. 


AUDIT OBJECTIVE AND SCOPE 


Our audit objective was to assess whether the Ministry had established satisfactory systems 
and procedures to fulfil its statutory responsibilities with respect to operators of school purpose 
vehicles and to contribute to the safe transportation of students. 


Our audit was performed in accordance with the standards for assurance engagements, 
encompassing value for money and compliance, established by the Canadian Institute of 
Chartered Accountants, and accordingly included such tests and other procedures as we 
considered necessary in the circumstances. 


We visited the Ministry’s head office and the three district offices that were responsible for 
most of the areas covered by the school boards we had visited during our audit of the Ministry 
of Education’s pupil transportation grants. Our observations and conclusions were based on 
criteria that were agreed to by the Ministry’s senior management. Our work was conducted 
from January to June 2000. 


The Ministry’s Internal Audit Services Branch had not done any recent work on which we 
could rely to reduce the extent of our work. 


OVERALL AUDIT CONCLUSIONS 


The Ministry can and should strengthen its systems and procedures for ensuring that operators 
of school purpose vehicles comply with legislative and regulatory safety requirements. In 
particular, the Ministry did not regularly update its policies to reflect changing conditions and 
risks, nor had it developed processes to capture the information needed to ensure that: all school 
buses are subject to being selected for inspection; those operator facilities and inspection 
stations posing the highest risk of non-compliance are selected for audit; and enforcement 
personnel carry out their inspection and audit activities in accordance with ministry policies. 


The Ministry had not sufficiently communicated the nature, extent and results of its 
enforcement activities to school boards and needed to coordinate efforts with them so that all 
safety risks are addressed and appropriate actions taken. 





Overall Ministry Response 


The Ministry is strongly committed to continuously improving the safety of all 
commercial vehicles operating in Ontario, including school buses and other 
school purpose vehicles. Over the last three years, the Ministry has 
established a regulatory regime for all commercial vehicles, including Sone! | 
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buses, that is among the most comprehensive and stringent in North 

America. The success of the Ministry’s school bus inspection program is 
demonstrated by the steady decline over the last few years of the out-of- 
service rates, the number of charges laid during inspections and, most 

importantly, collisions. We have also worked with the Ministry of Education 

_and Transport Canada to consult and communicate with stakeholders on 
school bus safety issues. — 


















| Nevertheless, we do recognize that improvements can be made to the 
Ministry’s monitoring/tracking systems for the bus inspection program to 

| assist enforcement to better identify and target higher-risk school bus 

operations. Proposed changes to supporting management information 

systems will strengthen the Ministry’s monitoring and reporting capability in 

this program area. — 





DETAILED AUDIT OBSERVATIONS 


STRENGTHENING ENFORCEMENT 
PROCEDURES 


One of the requirements of the Highway Traffic Act is that commercial carriers have their 
vehicles inspected and certified as to their mechanical fitness by ministry-licensed Motor 
Vehicle Inspection Stations (MVIS) every six months. (Large and mid-sized carriers are 
normally licensed to operate their own MVIS.) 


The Carrier Safety and Enforcement Branch discharges its legislated responsibilities through its 
program of vehicle inspections, audits of carriers’ facilities and audits of MVISs, which are 
carried out by enforcement staff at ministry district offices. Ministry statistics on collisions, pupil 
injuries and the rate of serious defects found during school bus inspections indicate a positive 
trend in pupil transportation safety and the mechanical fitness of school buses. However, as 
detailed below, we found several weaknesses in the Branch’s audit and inspection processes 
with respect to school purpose vehicles that should be corrected to help ensure the continuation 
of this positive trend. 


ESTABLISHINGASSURANCE OBJECTIVES 


The Ministry carries out its program of audits and inspections in order to obtain assurance that 
operators of school purpose vehicles are complying with legislative and regulatory safety 
requirements. No audit or inspection process can deliver absolute assurance regarding | 
compliance. There is always a risk that the process will fail to detect cases of non-compliance. 


This risk is inversely related to sample size: the bigger the sample, the smaller the risk. 
However, larger samples increase audit and inspection costs. Therefore, setting the tolerable 
level of risk is a key strategic decision for any audit or inspection process as it determines the 
size of the sample and therefore the amount of work to be done. This creates a link between 
assurance objectives and the resources required to meet them. 
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However, we found that the Branch had not specified the assurance objective for its program 
of inspections and audits. Consequently, the amount of work done was a function of budget 
rather than the budget being a function of the amount of work necessary to meet safety 
assurance objectives. Establishing assurance objectives would help the Branch determine and 
defend its resource requirements. 


DOCUMENTING AND REVIEWING METHODS 


The Branch had not documented the assumptions and risk analyses on which its program of 
audits and inspections was based. This documentation would facilitate the periodic review of 
the Branch’s audit and inspection methods and enable senior branch management to: 


* assess whether the Branch’s methods are appropriate and sufficient; and 


* adjust their methods to reflect opportunities and risks introduced by advances in technology 
and systems or by changes in the transportation operating environment, such as increased 
pressures on school bus operators to reduce costs or shortages of trained drivers. 


SAMPLE SELECTION 


Effective sampling for inspection and audit activities requires that all buses and facilities are 
subject to being selected for inspection or audit. Thus, for example, school bus operators should 
not be able to reduce the chances of their least fit buses being inspected by assigning them to 
remote routes. However, we found that the Branch’s sample selection procedures did not 
ensure that all school buses were subject to inspection. 


We also found that both the efficiency and effectiveness of the Branch’s sampling could be 
improved by using information technology to incorporate risk-related data into the selection 
process and thereby focus inspection/audit activities on the buses, facilities and MVISs that 
have the highest risk of non-compliance with safety legislation. 


Collectively, the Ministry’s various information systems—the Commercial Vehicle Operators 
Registration (CVOR) system, the vehicle licensing system, the driver licensing system and the 
MVIS licensing system—have much data that would help the Branch identify high-risk 
operators for inspection or audit. We noted, for example, that: 


* — District offices do not have reports that list all the school buses in their jurisdictions or that 
include risk-related data such as vehicle age. Such records would enable district offices and 
the Branch to ensure that all school buses operating in the province are subject to 
inspection and that inspection efforts are focused on the highest-risk vehicles and 
operators. However, the Branch does not have procedures and programs to extract the 
required data from the Ministry’s systems. 


* The details of vehicle inspections, such as the date of inspection, the date that the vehicle’s 
MVIS certificate was issued and the licence number of the MVIS were not recorded on 
the CVOR system. Such data would enable the Branch to rank MVISs based on the 
number of defects found in recently certified vehicles and thereby help identify higher-risk 
MVISs. The data would also enable management to monitor whether enforcement officers 
are focusing their efforts on those higher-risk MVISs. 


The Branch has a system, the Carrier Rating System (CRS), to assist in identifying unsafe 
commercial carriers for facility audits and other actions. The CRS assigns penalty points to 
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carriers with respect to accidents, convictions for violations of the Highway Traffic Act and 
inspection violations, and it maintains a running point total for the latest 24 months. The CRS 
compares each carrier’s points to an established standard for fleets of that size. A warning 
letter is sent when the accumulated points reach 35% of the standard, a facility audit is : 


performed at 65%, and suspension and cancellation of a carrier’s registration are considered at 
100%. 


However, the Ministry had not verified that the CRS standards were appropriate and working 
as intended for school bus operators. For example, we noted that none of the three facility 
audits of school bus operators that were conducted during 1999/2000 by the district offices we 
visited had been flagged for any action by the CRS, including that of one operator that scored 
poorly on its vehicle inspections and failed a subsequent facility audit. We also found that some 
inspections of school buses had not been recorded on the CRS. Incomplete information 
increases the risk that the CRS will fail to flag unsafe operators for action. In response to this 
problem, the Ministry implemented procedures for the CRS to contain all school bus inspection 
results from April 1, 2000 onwards. 


REPORTING AND FOLLOW-UP 


The reporting on inspections and audits related to school bus operators by district offices to 
branch management was not sufficient to enable management to ensure that enforcement 
personnel were adhering to the Branch’s policies. We noted that branch management did not 
have information about such matters as the timing of inspections and audits throughout the year, 
the number of vehicles inspected for each MVIS and summary information on audits and 
inspections for each school bus operator. We also noted that the monitoring done by head office 
was insufficient to detect differences that existed in the sample selection methodology and 
inspection and audit practices among district offices. These differences had not been approved 
by branch management. 

In addition, procedures had not been established to follow up on problems uncovered during 


facility audits. Consequently, the Branch did not have evidence that school bus operators had 
taken timely corrective action to resolve problems identified during facility audits. 





Recommendation 


To obtain reasonable assurance that school bus operators are complying 
with legislative and regulatory requirements, the Ministry should: 


¢ establish assurance objectives for its audit/inspection program and 
- determine resource requirements based on these objectives; 
* document and periodically assess whether its audit and inspection 
methods are appropriate and sufficient; 
e make greater use of information systems technology to better focus audit 
and inspection activities on high-risk operators; 
* establish reporting and monitoring procedures that allow management to 


ensure enforcement personnel throughout the province ae Seas 
audit/inspection activities in accordance with the Branch’s policies; an 
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¢ establish follow-up procedures to verify that school bus operators take 
timely action to correct problems detected by audits of facilities and 
Motor Vehicle Inspection Stations (MVISs). 


Ministry Response 


The Ministry is pleased with the year-over-year reductions in bus-related 
collisions, fatality rates and out-of-service rates, as well as an increase in 
overall compliance rates. We are confident that our various programs, 
including inspection of over 2,800 school buses by ministry enforcement staff 
last year, are promoting a high level of compliance by school bus operators in 
Ontario. Buses placed out of service for critical defects have declined from 
7% in 1997/98 to 5.6% in 1999/2000. Revisions to our information systems will 
enhance our ability to track and monitor these enforcement activities at a 
local level and to better identify high-risk operators. 








The Ministry will reaffirm policy direction and procedural expectations with 
field staff. In addition, we will take steps to monitor staff activities through 
routine performance reviews. Specifically, the Ministry will consider a 
sampling protocol that will provide a high level of assurance that school bus 
operators are meeting their legal obligations. The Ministry will also 
periodically assess whether its procedures continue to be effective and 
efficient in light of changing conditions. 


The Ministry will also take steps to obtain an inventory of school purpose 
vehicles and ensure that each ministry enforcement district has a record of 
vehicles in its area. 





Last year, the Ministry obtained the legislative authority to require operators 
to provide proof that vehicle defects found during inspections have been 
corrected and is currently developing a defect repair verification system. The 
Ministry will consider employing a defect repair verification system for 
problems found during facility and MVIS audits as well. We also note that a 
carrier that fails an audit is rated “conditional” and remains conditional for a 
minimum of six months and until it is able to pass an audit. Ratings 
information is available to the public, and we feel that the negative impact of a 
conditional rating on marketing efforts acts as an incentive to comply with 


| Pequlrements. a 











ESTABLISHING A PROTOCOL FOR 
COOPERATION WITH SCHOOL BOARDS 


Both school boards, with respect to their civil liability, and the Ministry, with respect to legislated 
requirements, have responsibilities for ensuring that school bus operators conduct their 
operations in a safe manner. Given this shared responsibility for safety, knowledge of the extent 
and results of compliance work done by the other party would assist both in minimizing the risk 
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of gaps in the overall safety framework. However, a protocol to share such information 
between school boards and the Ministry has not been developed. 


At the three district offices we visited, we noted that there were no procedures to communicate 
the results of their vehicle inspections and facility audits to school board transportation 
managers. Only one of the district offices was aware of the 1982 directive on Ministry 
communication to school boards, which states that school boards should be notified when a 
vehicle used on their routes is pulled out of service as a result of a vehicle inspection and that 
the inspection reports may be forwarded to the boards. The other two offices did not notify 
boards about out-of-service vehicles, and staff at one of these offices stated that they would 
not give reports to school boards. However, the school board transportation managers we spoke 
with stated that they would find such information useful. 


Similarly, although many school boards have established, through contracts, a right to audit the 
adequacy of the safety procedures of their carriers, none had procedures for sharing the results 
of such work with the Ministry’s local district office. Also, school boards did not share with the 
Ministry the complaints about unsafe practices or driving that they receive from time to time. 

























Recommendation | 


To better ensure that timely action is taken to enforce pupil transportation 

| safety requirements, the Ministry should work with school boards to develop 
a protocol that sets out the expectations of each party and establishes 
procedures to coordinate activities and exchange the results of their work. 


| Ministry Response 

The Ministry provides stakeholders with information about commercial 
vehicle safety requirements and its related programs and activities in a 
variety of ways. With respect to school boards specifically, the Ministry sent a 
letter to the Ministry of Education and public and Catholic school board 
associations encouraging boards to use safety ratings in selecting carriers 
for student transportation. 


The Ministry will take immediate steps to improve the level of communication 
with school boards. The Ministry has written to the Ministry of Education and 
offered to discuss with the Ministry of Education and school boards: 


¢ linking relevant Web sites to provide quicker and easier access to school 
bus safety information; 

¢ whether the Ministry can assist in reviewing/developing contract 
provisions that consider a carrier’s safety record; 

¢ sharing inspection/audit information; 

¢ preparing articles and sharing information through relevant school board 
newsletters, trade media, and so forth; and 

° whether school boards can provide useful information to the Ministry to 

better focus its enforcement resources. 
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CHAPTER FOUR 


Follow-up of 
Recommendations in the 
1998 Annual Report 





Since 1993 it has been our practice to make specific recommendations for corrective action by 
ministries and agencies and, two years after publication of the recommendations in our report, 
to follow up on the status of action taken. This chapter provides some background on the audits 
comprising the Value for Money Chapter of our 1998 Annual Report as well as the current 
status of implementing the recommendations made. We are pleased that in many cases our 
recommendations have been either fully or substantially implemented. However, in several 
cases, progress has been slow or is ongoing. In cases where the recommendations have not 
been implemented, or are still in the process of implementation, a brief description of the current 
status of action taken by the ministries is provided. 


Ministry of Community and Social 
Services: 

Business Transformation Project/ 
Common Purpose Procurement — 
3.01 


BACKGROUND 


In 1995/96, the Ministry of Community and Social Services initiated the Business 
Transformation Project to develop new business processes and technologies that would support 
the transformation of the Family Benefit and General Welfare Assistance programs into the 
Ontario Works Program and Ontario Disability Support Program. The Business Transformation 
Project is to provide technologies for single-tier delivery of the new social assistance and 
employment initiatives and, in doing so, replace the interim computer systems of Caseworker 
Technology and Ontario Works Technology, as well as the outdated computer systems, the 
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Comprehensive Income Maintenance System (CIMS) and the Municipal Assistance 
Information Network (MAIN). 


Under this initiative, the Ministry entered into a Common Purpose Procurement (CPP) 
agreement with Andersen Consulting on January 27, 1997, for the development and 
implementation of the business processes and technologies inherent in the new social assistance 
system that was to be put in place through the Business Transformation Project. The term of 
the agreement was for four years plus the possibility of a one-year extension, subject to mutual 
agreement, to be decided upon by the end of the second year of the agreement. 


Our 1998 audit assessed whether the Ministry had: 


¢ clearly established the appropriateness of the CPP process for its Business Transformation 
Project and had followed a reasonable and fair competitive selection process in awarding 
the agreement to Andersen Consulting; and 


¢ demonstrated due regard for economy and efficiency in the contract terms agreed to and in 
the administration of the work performed to the end of our audit field work in February 
1998. 


We concluded that the Ministry had not clearly established the appropriateness of the CPP 
process for the Business Transformation Project. We also noted that although the Ministry had 
followed the CPP principles in selecting Andersen Consulting as the successful vendor, in doing 
so, it could not demonstrate that it had selected the most cost-effective proposal or that the 
accepted proposal would result in value for money spent. 


We also concluded that the Ministry had not demonstrated due regard for economy and 
efficiency in the contract terms agreed to or in the administration of the work performed to the 
end of our audit field work in February 1998. 


As aresult of our observations, we made a number of recommendations intended to help the 
Ministry in assessing the appropriateness of any future Common Purpose Procurement 
agreements as well as for improving the Ministry’s administration of the current CPP 
agreement with Andersen Consulting. 


Subsequent to our 1998 report, and in part because of its own concerns with respect to the 
Business Transformation Project, the Ministry retained a consulting firm to conduct an 
assessment of alternative ways to proceed with the remaining phases of the Business 
Transformation Project. The Ministry identified two options to be assessed: 


* Option one: proceed with the existing CPP vendor under a modified agreement; and 


* Option two: terminate the existing agreement and proceed with an open procurement 
process to secure a qualified service provider to perform the remaining work under a fixed 
price—fee for service” agreement. 


The consulting firm recommended that the Ministry select option one and enter into time-limited 


discussions with Andersen Consulting to arrive at a mutually satisfactory arrangement for 
successfully completing the remaining work. 


In that regard, we also note that the Standing Committee on Public Accounts held bearings in | 
December 1999 into the report on the Andersen Consulting agreement from the 1998 Provincia 
Auditor’s Annual Report and an interim follow-up report issued at that time. As a result i | 
these hearings, the Committee passed a motion that, among other things, recommended that: 


ite 
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* the $180-million cap should not be increased, and expenditures excluded from the cap 
should be minimized; 


* the Ministry should develop an auditable system of benchmarking to ensure that initiatives 
taken by the Ministry prior to the Business Transformation Project are not incorrectly 
attributed to Andersen Consulting; 


* Andersen Consulting’s billing rates should be reduced; and 


* no further payments should be made to Andersen Consulting until the primary objective of 
implementing new technology is met and overall benefits exceed costs. 


On April 19, 2000, the Ministry and Andersen Consulting signed an Amendment Agreement to 
the original Master Agreement for the Business Transformation Project. The changes that 
were made through the Amendment Agreement as they relate to each of our recommendations 
are detailed below. 


We also note that at March 31, 2000, the project’s cost pool totalled $146.7 million ($117.4 
million Andersen Consulting, $29.3 million Ministry) and the benefit pool totalled $116.2 million, 
so the cost pool exceeded the benefit pool by $30.5 million; payments to Andersen Consulting 
totalled $95.6 million. 


CURRENT STATUS OF RECOMMENDATIONS 


The Ministry has not entered into any other CPP agreements to date and therefore did not have 
an opportunity to implement our recommendations relating to any future CPP agreements. 
These included recommendations for: 


* Choosing Common Purpose Procurement; 
* Competitive Selection Process; 

¢ Maximum Contract Payments; and 

* Cost Pool—Interest Charges. 


The Ministry has taken actions on many of our remaining recommendations through the current 
agreement renegotiation process with Andersen Consulting, as detailed below. However, 
notwithstanding the Ministry’s commitment to implement these recommendations, as noted in 
our 1998 report, its ability to do so has been affected in many instances by the terms of the 
original agreement and the ability of Andersen Consulting and the Ministry to negotiate and 
accept changes thereto. 


The Ministry substantially implemented our recommendations relating to the following areas: 
* Cost Pool—Out-of-Pocket Expenses; 

* Additional Consulting Assistance Acquired; and 

* Quality Council (council disbanded) 


The status of the remaining recommendations on the agreement between the Ministry and 
Andersen Consulting is as follows: 
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EXCLUDED COSTS 


Recommendation 


In order to more effectively estimate and control overall ministry payments for future 
Common Purpose Procurement projects, the Ministry should minimize the number of items 
excluded from maximum payment amounts. When items are excluded from maximum 
payment amounts, the items and the circumstances under which costs for them ought to 
be incurred should be clearly defined and estimated. 


Current Status 


Although the revised agreement did not alter the maximum fee cap of $180 million payable to 
Andersen Consulting, it did expand on the circumstances under which payments may be made 
outside of the fee cap. In addition to the original costs to be excluded from the fee cap, as noted 
in our 1998 report, the revised agreement provides for the following additional costs to be 
excluded from the fee cap: 


¢ Out-of-Scope Costs: These costs include work requested by the Ministry that is not 
included in the scope of the project as defined by the Service Delivery Model (detailed blue 
print) approved in December 1998. Such work is to be billed at Andersen Consulting rates 
as renegotiated in the Amendment Agreement. 


¢ Delay Costs: Between January 1, 2000 and the completion of the Service Delivery Model 
pilot, delay costs will be charged to a delay pool at the rate of $75,000 for each additional 
project work day required by Andersen Consulting to complete tasks identified in the 
critical path that result from a ministry-caused delay. For each such day after the 
completion of the pilot, the charge is $37,500. 


However, such costs are only to be paid if Andersen Consulting’s total costs exceed the 
maximum fee cap as a result of these delays and in any case are not to exceed $10 million. 
In addition, Andersen Consulting will not be entitled to any delay costs if the project is 
completed by the expiration of the Amendment Agreement on January 26, 2002. 


* Ministry Tasks Completed by Andersen Consulting: In order to prevent a delay to the 
schedule, the Ministry may request Andersen Consulting to perform tasks that would 
otherwise be the responsibility of the Ministry. Such work is to be billed at Andersen 


Consulting’s renegotiated rates. 
The following table provides a summary of the Ministry’s current estimated costs for each of 
the items excluded from the agreement fee cap as well as costs approved and incurred to 
March 31, 2000. 
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Summary of Costs Incurred and Budgeted as of March 31, 2000 for Items Not Included 
Under the Maximum Contract Fee Cap 


Costs Charged as of Mey 
Costs Outside of the Maximum Contract Fee g Estimated 
March 31, 2000 Coste 


Production Support, Help Desk, me 
Application Maintenance $1.3 million 














$23 million 




































Hardware and Purchased Third-Party Software $0 
Out-of-Scope Costs $275,000 $5.7 million* 













Delay Costs 


Ministry Tasks Completed by Andersen Consulting 








n/d = not yet determinable by the Business Transformation Project 
* up to $20 million requested and approved by MBS for the project 


Source: Ministry of Community and Social Services data 


COST POOL 
BENEFIT DISTRIBUTION—CHARGEABLE RATES 


Recommendation 


Future Common Purpose Procurement agreements should ensure that project savings are 
distributed equitably, based on the relative contributions of ministry and consulting staff, 


which will not necessarily correlate with the relative salary levels of ministry staff or the 
billing rates of consulting staff. 


Current Status 


As a result of the renegotiation process, the rates charged to the project for Andersen 
Consulting’s staff time were reduced as of January 1, 2000 and fixed for the remainder of the 
agreement. The change in rates is a noteworthy improvement from the previous arrangement 
that had permitted Andersen Consulting to charge fees at the standard published billing rates, 
which could be unilaterally increased by Andersen Consulting from time to time. However, the 


reduced rates now charged are still significantly higher than the rates charged for ministry staff 
doing comparable work. 


The table below compares the rates proposed by Andersen Consulting in response to the 1995 
request for proposals, the rate charged at the time of our audit in December 1997 and the 
current renegotiated rates for both Andersen Consulting and ministry staff time. 
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Comparison of Andersen Consulting’s 1995 Proposed Rates with Actual Rates at 
December 31, 1997 and January 1, 2000, and with Ministry Rates at January 1, 2000 
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Partner/ 
Associate Partner/ 
ADM/ 


300-400 
Project Director 


200-300 


Consultant 150-250 
Analyst 70-140 








530-575 











335-472 
230-325 
105-250 





















Source: Ministry of Community and Social Services data 


As a result of the continuing significant differences between the rates charged for Andersen 
Consulting and ministry staff time, our concern remains that under the renegotiated agreement 
Andersen Consulting is still receiving a disproportionate amount of the benefit pool in relation to 
its work effort. 


INCOMPLETE MINISTRY COSTS 


Recommendation 


To ensure that the benefits of the Ministry's Business Transformation Project are fairly 
distributed, the Ministry should include all of its costs related to that project in the 
project cost pool. 


Current Status 


In response to our 1998 audit observations and recommendation, the Ministry added 
approximately $180,000 in costs associated with Change Reporting systems changes that had 
been previously excluded from the cost pool. 


However, the ministry staff costs that we identified in 1998 for manual file reviews conducted 
under the Change Reporting task order were not added to the cost pool, even though the related 
benefits were included in the benefit pool. Similarly, we found that the ministry staff costs for 
implementing the new Consolidated Verification Process task order were also not added to the 
cost pool, even though those benefits were also added to the benefit pool. 


We also noted that in March 2000, a ministry review identified approximately 45 additional 
ministry staff members who potentially contributed to the Business Transformation Project but 
whose related time was not included in the cost pool. Although they were advised to report 
their hours spent on the project for inclusion in the cost pool, to July 2000 their costs had ak 4 
been added to the cost pool. We understand that the Ministry subsequently completed a process 
that resulted in the salary costs of 31 staff members being added to the cost pool. 
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It remains our view that the Ministry’s contribution to the cost pool continues to be understated, 
with the result that fewer benefits are being allocated to the Ministry, and to the taxpayer, than 
should be the case until the $180-million payment cap is reached. 


BENEEIT4OOE 


Recommendation 


The Ministry should ensure that if additional work is to be incorporated into the current 
or future Common Purpose Procurement projects, the benefits of that work as compared 
with the benefits of other alternatives are clearly established. 


Current Status 


Since the time of our original audit in 1998, the Ministry has approved task orders for three 
additional early opportunity initiatives: Tax Tables; Consolidated Verification Process; and 
Disability Determination. As of March 31, 2000, the benefits crystallized for each of the early 
opportunity initiatives and the related costs incurred are as detailed in the table below. 


Total Benefits and Costs for Task Orders as at March 31, 2000 


Andersen 

Benefits Consulting 
($ 000) Costs 
($ 000) 


Ministry 
Costs 
($ 000) 


Task Orders 


Early Opportunity Task Orders 
Change Reporting | 30,569 
Tax Tables 12,564 


Consolidated Verification Process 67,121 
= Disability Determination ay 5,930 
Total 116,184 
All Other Task Orders 0 
Total All Task Orders 116,184 

















Source: Ministry of Community and Social Services data 


The Ministry’s Internal Audit Services conducted a review of the Consolidated Verification 
Process initiative and found that, contrary to our 1998 recommendation, the Ministry had not 
identified or assessed alternative ways of proceeding with this work. Instead, the Ministry made 
a business decision to proceed with this work under the agreement. 


Given the disproportionate relationship between benefits crystallized and paid to Andersen 
Consulting and costs incurred, it remains our view that alternative ways of proceeding with this 
work should have been identified and assessed. In addition, we question the advisability of 
having paid Andersen Consulting over $95 million to March 31, 2000 given that the project was 
significantly behind its original schedule. We also note that, unless otherwise agreed to, the 
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agreement did not require any payments to be made until such time as total benefits exceeded 
total costs, which had not occurred to that date. Since the Ministry had agreed to make these 


payments, it was not in a position to alter this payment arrangement as recommended by the 
Standing Committee on Public Accounts. 


BENEFIT MEASUREMENT 


Recommendation 


To ensure that task order metrics fairly account for the relative contribution of savings 
by each partner in the Business Transformation Project, the Ministry should ensure that: 


* amounts included in the benefit pool are accurately determined; and 


¢ financial savings attributable to ministry staff adhering to previously established 
policies and procedures are not included in the project's benefit pool. 


Current Status 

Case terminations occur for various reasons, including changes in economic conditions, changes 
in policies, as well as changes in administrative practices. As a result of these multiple effects, 
the number of cases terminated as a result of changes in administrative practices, such as the 
Consolidated Verification Process (CVP), cannot be determined with absolute certainty. 
Instead, the incremental effect of the CVP initiative, for example, was estimated and included 
in the benefit pool based on a statistical model designed to obtain a 99% confidence level that 
benefits were not overstated. 


Our concern remains that much of the benefits so determined could and should have been 
achieved had ministry staff adhered to the existing policies and procedures for determining 
recipient eligibility and implemented recommendations made in previous Provincial Auditor 
reports on the social assistance systems. As such, it remains our view that these benefits are 
not clearly attributable to the changes inherent in the CVP initiative. 


However, we understand that the inclusion of this initiative in the Business Transformation 
Project had already been agreed to by both parties, and therefore, the Ministry was not ina 
position to exclude this initiative and the resulting benefits from the Project. 


CURRENT PROJECT STATUS 


Recommendation 


The Ministry should take the steps necessary to ensure that the work under the 
agreement with Andersen Consulting supports the delivery of the Ontario Works Program 
and the Ontario Disability Support Program with the revised business processes and 
technology solutions at the earliest opportunity. 


Current Status 


Originally planned for rollout completi 
Delivery Model is currently as follows: 


¢ Design, Build and Test Phases: to conclude in January 2001 


on by June 1999, the revised timetable for the Service 
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¢ Pilot Testing: February 2001 to April 2001 


¢ Rollout: February 2001 to January 2002 (that is, completion two and a half years behind 
original plan) 


Ministry of Community and Social 
Services: | 
Ontario Works Program — 3.02 


BACKGROUND 


First announced in June 1996, the Ontario Works Program has as its objective to provide 
financial assistance to participating individuals while they become self-sufficient and 
contributing members of their community by following the shortest route to a paid job. 


The Ontario Works Program provides for employment assistance under the three following 
components: 


* Employment Support: This component helps participants become job-ready and supports 
their shortest route to paid employment through job search assistance or participation in 
basic education or job-specific skills training. 


* Community Participation: This component enables participants to contribute to the 
betterment of their community while receiving social assistance and to gain valuable work 
experience, employment-related skills and access to networks that will help them move into 
the paid workforce. 


¢ Employment Placement: This component places job-ready participants into unsubsidized, 
competitive employment and may include supporting participants interested in self- 
employment. 


At the time of our audit in March 1998, participation in the Ontario Works Program was 
mandatory for most recipients of the former General Welfare Assistance Program. During the 
1997/98 fiscal year, approximately 251,000 recipients received General Welfare Assistance 
benefits totalling $1.76 billion. 


The Ontario Works Program is now delivered across the province through 47 municipal delivery 
agents (63 in 1998) representing large municipalities or groupings of smaller municipalities. 
Municipal delivery agents are accountable to the Ministry and report to one of the Ministry’s 
nine regional offices. To facilitate the implementation of the Ontario Works Program, each 
municipal delivery agent was expected to complete a business plan for the first three years of 
the Program. 


Our 1998 audit assessed whether the Ministry’s administrative procedures for the Ontario 
Works Program were adequate to ensure that: 


* transfer payments to municipal delivery agents were reasonable and satisfactorily 
controlled; and 
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* services provided by municipal delivery agents were monitored and assessed to determine 
whether they were meeting the Ministry’s expectations. 


We concluded that transfer payments approved for municipal delivery agents were reasonably 
controlled in that they were directly related to the amount of services provided. We also 
concluded that the Ministry needed to improve its monitoring and assessment of services 
provided by municipal delivery agents to Ontario Works Program participants to determine 
whether they were meeting the Ministry’s expectations. As a result, we made a number of 
recommendations for improvement. 


CURRENT STATUS OF RECOMMENDATIONS 


The Ministry has substantially implemented our recommendations related to the following areas: 
¢ Status of the Ontario Works Program; 

¢ Program Registration and Participation Agreements; 

¢ Program Monitoring; and 

¢ Measuring Program Effectiveness. 


With respect to our remaining recommendation, the status of action taken is as follows: 


INFORMATION SYSTEMS 


Recommendation 


The Ministry should ensure that the available information systems are adequately meeting 
the needs of the Ontario Works Program and the municipal delivery agents. 


Current Status 

The Ministry has advised us that improvements are continually being made to the existing 
interim Ontario Works information system to address priority legislative changes and to meet 
the needs of the Program and its municipal delivery agents. Changes already made include the 
addition of new data elements that enhance reporting capabilities to reflect new Ministry 
initiatives as well as to assist in tracking program performance and funding. 


The Ministry is currently in the process of implementing remote access to the Ontario Works 
information system for selected ministry program staff. The Ministry still intends to replace the 
interim Ontario Works information system with the new Service Delivery Model Technology 


System, which is targeted for implementation in 2002. 


Special Report: Accountability and Value for Money 





Ministry of Education and Training: 
Acquisition and Management of 
Elementary and Secondary School 
Facilities — 3.03 


BACKGROUND 


In 1998, school boards operated over 5,100 schools, which represented a replacement cost of 
approximately $26 billion. Under the then Ministry of Education and Training’s student-focused 
funding model, school boards received over $1.5 billion in pupil accommodation grants for the 
1998/99 fiscal year ($1.7 billion in 1999/2000). This amount was intended to cover the cost of 
cleaning, maintaining and renovating schools and of constructing or acquiring new schools. 


In 1998, we assessed whether satisfactory systems and procedures had been established at the 
Ministry and selected school boards for the acquisition and management of school facilities, 
including compliance with related regulations and policies, and whether information systems 
provided adequate support for management control, decision making and performance 
reporting. 


CURRENT STATUS OF RECOMMENDATIONS 


The Ministry had made significant progress in addressing our recommendations but required 
more time to complete the actions it intends to take to address all of our recommendations. 


In early 2000, the Ministry implemented an Accountability Framework and Reporting Process 
for pupil accommodation grants that requires school boards to report annually to the Ministry, 
trustees and the public on: the condition of each school and its equipment; the nature and cost 
of each new school construction and renewal project; the cperating expenditures for each 
school; the user survey results for each school regarding its cleanliness and comfort; and the 
board’s long-term enrolment forecasts and its plans for meeting the accommodation needs of its 
students. Boards are expected to submit reports for the year ended August 31, 2000 by 
December 31, 2000. 


In June 2000, the Ministry completed the development of a School Facilities Inventory System 
(SFIS) that contains detailed information about each board’s properties and buildings and will 
eventually also contain school-specific information that will be obtained from each school 
board’s annual report. The SFIS allows school boards to update their information electronically 
and to review the information submitted by other boards and compare it to their own. 


These significant actions will provide the information that school boards and the Ministry need 
to ensure the effective management of school board properties. The Ministry now intends to 
allocate resources to implementing our other recommendations, several of which required the 
information that school boards will now be providing under the Accountability Framework. The 
status of each of our recommendations as at July 31, 2000 is discussed below. 
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USE OF EXISTING FACILITIES 


Recommendation 
To promote the efficient and economical use of school facilities, the Ministry should: 
* establish procedures to verify the existing capacity of schools; 


* encourage and assist school boards to evaluate the feasibility of various approaches 
to increasing capacity and improving the utilization of facilities; 


* provide further guidance to boards to help them identify and dispose of surplus 
schools more expeditiously; and 


* require boards to justify decisions to build new schools rather than purchase 
available surplus schools from neighbouring boards. 


Current Status 


In 1999, a team of ministry and seconded school board personnel reviewed and corrected the 
facility information that boards submitted for the SFIS and that the Ministry relies on to 
determine grant entitlements. The review was repeated in the first half of 2000 to validate 
updated information. Now that these steps have been taken, the Ministry intends to select a 
sample of schools across the province to verify the accuracy of the SFIS information, but it had 
not yet set a target date for completing this project. 


The Ministry also intends to use school enrolment and capacity information to identify schools 
with high utilization rates and then determine the good practices that were employed to achieve 
these high utilization rates and share these findings with all boards. 


School utilization rates will also be monitored in relation to the long-term accommodation plans 
submitted by school boards to determine how boards plan to address imbalances between 
enrolment and capacity in their schools. The Ministry will continue to provide guidance to 
boards, when requested, on the information they should prepare in connection with school 
closures and disposals but will not establish further requirements unless it identifies problems 
from its ongoing analysis of school plans and utilization data. 


LONG-TERM PLANNING FOR ECONOMIC USE OF 
FACILITIES 


Recommendation 
To help ensure that pupil accommodation decisions represent the best long-term value, 
the Ministry should coordinate research on: 


° the relationship, if any, between school utilization rates and student achievement; 
° the relative operating costs of permanent classrooms and portables; and 


° the methods for preparing reliable long-term enrolment forecasts. 


Current Status 


Once school condition and other accommodation ‘eC eh 
compare this information to the performance of students on tests administered by t 
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Quality and Accountability Office and determine what relationship, if any, exists between 
accommodation characteristics and student achievement. 


The Accountability Framework requires boards to report information on facility operating costs 
on a per-school basis. The Ministry intends to begin its study of the operating costs of 
permanent classrooms and portables once this information is received. 


The Ministry has developed workshops through which it shares information about its provincial 
forecasting methodologies and its board-level enrolment forecasts with school boards. The 
Ministry was still considering options on how to ensure that boards’ long-term enrolment 
forecasts at the individual school level are reliable. 


DEVELOPMENT PARTNERSHIPS 


Recommendation 


To help ensure that school boards take advantage of opportunities to reduce costs 
through development partnerships, the Ministry should: 


* require boards to summarize and report on their efforts to find partners for each new 
development and, where independent developments take place, to explain why a 
partnership is not feasible; and 


* encourage other ministries to include conditions or incentives in their grant programs 
requiring recipients to enter into cooperative or development partnerships with 
school boards where feasible. 


Current Status 


As the Ministry responded in 1998, it wants to assess how well the existing funding formula and 
reporting requirements work in encouraging school boards to enter into development 
partnerships before imposing additional reporting requirements. 


The SFIS now provides the Ministry with information on schools that were built in partnership 
with others. Once school operating costs are also entered into the SFIS, the Ministry intends to 
identify and investigate the capital and operating cost savings achieved by shared facilities 
relative to the costs of stand-alone facilities. The Ministry will then share its findings with all 
boards. 


LIFE-CYCLE COSTING 


Recommendation 


To assist boards to better manage pupil accommodation costs over the long term, the 
Ministry should: 


¢ help boards to evaluate systems that support a life-cycle approach to accommodation 
spending decisions and to share implementation and maintenance experiences with 
each other; and 


¢ coordinate the collection and sharing of performance data for materials and 
equipment and provide guidance to boards regarding the preparation of business 
case analyses to support major purchase decisions. 
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Current Status 


Implementation of the first aspect of this recommendation was being addressed in conjunction 
with our recommendation for monitoring operating costs and results, the status of which is 
discussed below. 


With respect to the second aspect of this recommendation, the Ministry intends to analyze 
information about school operating costs, conditions and user satisfaction, and, over time, 
identify schools that achieve high condition and user satisfaction ratings at a lower long-term 
cost than their peers. The Ministry intends to investigate the reasons for this better 
performance, including whether it is attributable to the use of certain building materials and 
equipment. The Ministry also intends to schedule workshops to enable board facility 
management officials to share information on their experience with different products. 


MONITORING OPERATING COSTS AND RESULTS 


Recommendation 


To ensure that boards acquire and implement the information systems needed to manage 
their facilities and costs and to report on results, the Ministry should: 


¢ provide guidance to boards regarding the information and analysis required for 
effective facility management and related results reporting; 


¢ establish a mechanism for enabling boards, which have implemented systems to 
address facility management information needs, to share their experiences with other 
boards; and 


* examine options to minimize the cost of the substantial investment in management 
information systems that boards must make. 


Current Status 


The Ministry advised that it will implement this recommendation in cooperation with the Pupil 
Accommodation Review Committee. This Committee, composed of three ministry and 14 
school board representatives, continues to advise the Ministry on pupil accommodation issues 
and to assist in designing and organizing related educational workshops for board personnel. 
The Committee conducted a survey in early 2000 of boards’ experiences with various asset 
management information systems, some of which support life-cycle costing. The survey found 
that 38% of boards did not have any asset management systems and that only 6% had 
specialized systems to assist them in managing all seven of the critical functions identified by 
the Committee. In view of the substantial system needs identified by this survey, the Ministry 
was investigating options to minimize the cost to boards of implementing such systems, including 


bulk purchasing. 


The Ministry, in conjunction with the Committee, also intends to provide boards with guidance 
and training on preparing business case analyses and collecting, analyzing, and reporting 
performance information about facilities management. The Ministry has acquired software that 
will enable it to provide boards with easy electronic access to training and workshop 
presentation materials when they are completed. 


eS ee 


avl 
Special Report: Accountability and Value for Money 





MAINTENANCE EXPENDITURES 


Recommendation 
To help ensure that school boards do not continue to defer needed maintenance, the 
Ministry should: 


« require the boards to conduct objective, reliable condition assessments of each 
school on a periodic basis and estimate the cost of needed repairs; 


« require the boards to include such information in reports to the trustees and the 
Ministry; and 


* monitor grant levels to assist boards to manage their assets prudently over the long 
term. 


Current Status 


The Accountability Framework requires boards to assess and report on the condition of each of 
their schools and to report the results of user satisfaction surveys. Once this information is 
obtained, the Ministry intends to use it to calculate a condition index for each board. The 
Ministry intends to monitor trends in the index to assess whether each board’s facilities 
management is effective and whether pupil accommodation grants are sufficient to fund a cost- 
effective maintenance and repair program. 


DOCUMENTATION OF OBJECTIVES AND PROCEDURES 


Recommendation 


To assist school boards in maintaining service continuity and evaluating the efficiency, 
economy and effectiveness of operating procedures, the Ministry should: 


* encourage school boards to document their objectives and the procedures to achieve 
them; and 


* support efforts to share ideas and conduct pilot projects that reduce costs and/or 
improve services. 


Current Status 


The Ministry expects boards to establish objectives whose achievement can be monitored 
through the condition assessments and user satisfaction surveys. 


The Ministry had identified and shared with school boards best practices for preventing mould 
in portables. It was considering how the approaches used to manage and share the lessons 
learned from the mould problem could be applied to other facility management issues. 


THE ACCOUNTABILITY FRAMEWORK 


Recommendation 


To help ensure that the facilities management information produced by school boards is 
reliable and to assist trustees in meeting their responsibilities for setting policies, 
monitoring performance and taking corrective action, the Ministry should: 
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* establish a framework for independently verifying school board grant determination 
and performance information; and 


* provide trustees with best practices and training materials on governance that 
include their role in overseeing facility management activities. 


Current Status 


The Ministry has arranged for school boards’ external auditors to provide assurance on the 
reliability of enrolment information, which is a key determinant of school board funding 
entitlements. The Ministry intends to verify a sample of performance and operating data in 
connection with its test of the reliability of school capacity information. 


Following the municipal elections in November, the Ministry intends to provide trustees with 
information on good governance practices and training materials to assist them in overseeing 
facility management activities. 


Ministry of Finance: 
Land Transfer Tax Program — 3.04 


BACKGROUND 


The Land Transfer Tax Act requires that purchasers pay a tax when an interest in ownership 
of land is transferred in Ontario. The tax is based on the value of consideration paid by the 
purchaser as sworn in an affidavit. Generally, the tax is paid when the land transfer is 
registered at one of the 55 land registry offices operated by the Ministry of Consumer and 
Commercial Relations under the terms of a Memorandum of Understanding entered into with 
the Ministry of Finance. 


The rate of tax varies from 0.5% of the value of consideration below $55,000 up to 1.5% on 
amounts exceeding $250,000. For single family residences only, the rate of tax increases to 2% 
on amounts exceeding $400,000. In addition, land transfer tax may be waived or refunded in 
whole or part for first-time home buyers who meet prescribed conditions. 


For the 1997/98 fiscal year, the province collected $544 million in land transfer tax from 
approximately 345,000 transfers in interest in land. For the same year, approximately 20,000 
purchasers received refunds or exemptions having a total value of $2.8 million and $21.3 million 
respectively. 

Our audit objective was to assess whether the Ministry had appropriate policies and procedures 
in place to ensure that the correct amount of land transfer tax was being collected, refunded or 
exempted in accordance with statutory requirements. 


We concluded that both the Ministry and the land registry offices had adequate procedures in 
place to ensure that the appropriate amount of land transfer tax was collected and deposited to 


the Consolidated Revenue Fund, and refunded or exempted based on the values of 
consideration and other information declared by purchasers. 
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However, improvements were required to ensure that declared values of consideration and 
other information were reasonable and, therefore, ultimately to ensure that the appropriate 
amount of tax was declared and paid. 


CURRENT STATUS OF RECOMMENDATIONS 


We were pleased to note that the Ministry was in the process of, or had already substantially 
implemented, all of our recommendations. These recommendations related to the following 
matters: 


¢ Assessing Information Provided—Training and Informational Materials; 


¢ Enforcement Activities—Audit Work Performed, Audit Revenues, Penalties and Fines, and 
Objections and Appeals; 


¢ Accounts Receivable; and 


¢ Refunds and Exemptions. 


Ministry of Health: 
Long-Term Care Community Based 
Services Activity — 3.05 


BACKGROUND 


The Long-Term Care Community Based Services Activity provides funding for homemaking 
and professional services for people at home who would otherwise need to go to, or stay longer 
in, hospitals or long-term care facilities. Funding is also provided to community support service 
agencies that assist frail elderly people and people with physical disabilities to live as 
independently as possible in their own homes. 


In 1998, we assessed whether the Ministry had adequate procedures in place to measure and 
report on the effectiveness of the Activity; ensure compliance with applicable legislation and 
ministry policies; and ensure resources were used with due regard for economy and efficiency. 


CURRENT STATUS OF RECOMMENDATIONS 


Recommendations relating to the following sections of our 1998 report have been substantially 
implemented: 


¢ Compliance—Timeliness of Reporting, and District Health Council Long-Term Care Plans 


The current status of the remaining recommendations is as follows: 
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EFFECTIVENESS 


COST-EFFECTIVENESS OF LONG-TERM CARE 
COMMUNITY SERVICES 


Recommendation 
To assist it in making improvements to long-term care community services, the Ministry 
should develop a system to measure and report: 


¢ the costs of long-term care community services provided to individuals; and 


¢ the relevant performance indicators for Community Care Access Centres. 


Current Status 
The Ministry informed us that: 


¢ Itremains committed to implementing a new information system for all Community Care 
Access Centres (CCACs). An Integrated Management System will be implemented over 
the next two or three years. This new system will provide information about services 
provided to each CCAC client. This information will be a fundamental component for 
analyzing costs and developing benchmarks. 


¢ Case management standards are being developed for community programs. The Ontario 
Case Managers Association and the Ontario Community Support Association received a 
grant to develop the standards. Their report is due this summer. 


* Through their Best Practices Project, the Ministry and CCACs are laying the foundation 
for further research in the areas of service standards and performance measures. 


INSPECTIONS OF LONG-TERM CARE COMMUNITY 
SERVICE AGENCIES 


Recommendation 

To ensure that long-term care community service agencies are complying with provincial 

standards and providing quality services efficiently and effectively, the Ministry should: 

¢ develop appropriate inspection procedures and conduct periodic inspections of 
agencies; and 

* investigate options to assess whether service agencies have successfully implemented 


quality management systems. 


Current Status | 
A tool for operational reviews of CCACs has been developed. The Ministry is also conducting 
a CCAC Program Review to determine the extent to which CCACs are meeting their mandate 
to ensure simplified access to long-term care community based services. The Review will focus 
on CCAC management and service delivery, as well as on the Ministry's program 
administration. The final report, which is expected in November 2000, will influence further 
refinements of the CCAC operational review tool. 
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COMPLAINT MONITORING 


Recommendation 


To ensure that action is taken to improve service, the Ministry should: 


* require Community Care Access Centres and other long-term care community service 
agencies to periodically submit statistical information on the number and types of 
complaints they have received and their resolutions; and 


¢ develop a formal process to record the receipt and resolution of complaints. 


Current Status 


In the summer of 1999, the complaints provisions of the Long-Term Care Act were 
implemented for CCACs. These provisions require CCACs to develop a process for reviewing 
complaints. Other long-term care community agencies will also be required to comply with 
these provisions. 


The Ministry also indicated that: 


¢ The tool developed for operational reviews of CCACs includes a section about complaints 
and appeals, including how CCACs monitor complaints to ensure that emerging issues are 
addressed. 


* One component of the CCAC Program Review is directed at service delivery. Key 
questions will address the monitoring and evaluation mechanisms in place at CCACs to 
assess the processes and outcomes of delivery and the use of these mechanisms in 
resolving quality of care issues. 


MANAGEMENT INFORMATION SYSTEMS 


Recommendation 


To help ensure the efficient and effective delivery of long-term care community services 
and to provide information to properly plan and manage service delivery, the Ministry 
should develop: 


¢ aplan with specific timeframes for implementing the Community Care Access Centre 
Information System Network; and 


* procedures to verify that submitted data are complete and accurate. 


Current Status 


In 1999, the CCAC Integrated Management System Council, with representation from CCACs 
and the Ministry, was put in place. The Council agreed to a common base technology 
infrastructure to be implemented in all CCACs. The Council will determine the appropriate 
course of action once business requirements for the new system have been defined and 
approved by both the Ministry and CCACs. It is anticipated that a modular system will be 
implemented in stages. Accomplishments to date include: 


¢ The Base Technology Infrastructure, Phase I, is underway. 


¢ Planning and scheduling for Phase II of the Base Technology Infrastructure are complete. 
Implementation is underway. 
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¢ The Business Process Analysis report is being finalized. 


¢ The Business Area Analysis—Business Requirements Sub-Project has been initiated. 
Recruitment of staff is underway. 


¢ The Common Assessment Tool Sub-Project has been initiated and related research is 
underway. 


¢ The Information and Referral Sub-Project has been initiated. Recruitment of staff is 
underway. 


COMPLIANCE 
SERVICE AGREEMENTS AND FINANCIAL REPORTING 


Recommendation 
To help ensure that service plans and budgets are equitable and appropriate for each 
long-term care community service agency, the Ministry should: 


¢ set timeframes for signing agreements and reviewing and approving budgets; and 


¢ develop benchmarks for unit costs for each type of service. 


Current Status 

The Ministry informed us that: 

* Amendments to the service agreement (comprising a legal contract, a budget and a service 
plan) were drafted based on negotiations with CCACs. The legal contract has yet to be 


finalized by the Ministry. The Ministry anticipates that the final report of the CCAC 
Program Review will influence further adjustments to the legal contract. 


* The Best Practices Project, a joint initiative of the Ministry and CCACs, has several 
components that will provide a foundation for the identification of best practices (for 
example, partnerships with the research community, updating of the care pathway 
inventory, identification and communication of clinical benchmarks). 


VERIFICATION OF SERVICES 


Recommendation 


The Ministry should establish procedures to verify on a test basis that long-term care 
community services were received and properly authorized. 


Current Status 
The Ministry advised us that: 
¢ In its template for operational reviews, the Ministry has included a review of CCACs’ 


policies and procedures on monitoring and verifying authorized services. 


* One component of the CCAC Program Review, which is underway, is directed at CCAC 
management. Key questions address the effectiveness of financial management practices 
and their consistency with the requirements of the Long-Term Care Act, the service 


agreement and ministry policies. 
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ELIGIBILITY 


Recommendation 


To better ensure that only eligible individuals receive long-term care community services, 
the Ministry should implement procedures to verify that service recipients have valid 
Ontario Health Insurance Plan numbers. 


Current Status 


The Ministry is currently assessing two options—on-line and dial-in verification—to verify 
Health Card number validity. The Ministry is also planning to give CCACs access to an 
interactive voice response system and look-up service hotline. Implementation is anticipated in 
the 2000/01 fiscal year. 


ECONOMY AND EFFICIENCY 
FUNDING FORMULA 


Recommendation 


To better ensure equitable funding and access to long-term care community services, the 
Ministry should: 


¢ establish a plan to eliminate inequities in funding and differences in service levels 
among districts; 


¢ ensure that its funding formula takes into account service needs, ongoing 
demographic changes and changes in the health care system; and 


* review the appropriateness of funding allocations between Community Care Access 
Centres and community support service agencies. 


Current Status 


In April 1999, following a review of the Equity Funding Formula with key stakeholders, the 
Ministry made two adjustments to the existing formula. In June 2000, the Ministry convened the 
Long-Term Care Community Equity Funding Review Committee. Its objectives are to: 


* continue the review of the equity funding formula initiated in 1998/99 by the Ministry; 


* re-examine the long-term care community services equity funding formula and determine 
what additional adjustment factors and/or improvements should be considered in the equity 
funding formula; and 


* review existing guidelines for allocating funds to programs within an area and recommend 
changes/improvements to the distribution of allocated government funds between services 
of Community Care Access Centres and those of other long-term care community service 
agencies. 
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ACQUISITION OF SERVICES BY COMMUNITY CARE ACCESS 
CENTRES 


Recommendation 


To help ensure that the request-for-proposal process used by Community Care Access 
Centres is meeting its objectives, the Ministry should: 


¢ develop and implement standardized methods that Community Care Access Centres 
can use to assess whether the quality of service requirements in their requests for 
proposals are being met; 


¢ evaluate its implementation; and 


* consider how often requests for proposals should be issued. 


Current Status 


A committee involving CCACs, service providers and provincial stakeholder associations 
continues to meet to support the effective implementation of the request-for-proposal (RFP) 
policy. The committee has developed best practices, such as site visits as an evaluative tool, 
specialty RFP models (for example, pediatric, palliative) and the managing of volume 
distribution. 


QUALIFICATIONS OF PERSONAL SUPPORT WORKERS 


Recommendation 


To better ensure that long-term care community service recipients are receiving quality 
services from properly trained and qualified workers, the Ministry should develop a 
formal plan along with specific timeframes for fully implementing the standards of the 
Personal Support Worker Training Program. 


Current Status 


Increased funding was allocated to service providers starting in April 1999 to upgrade workers 
to the new Personal Support Worker qualifications. A survey of service providers was 
conducted to determine the number of people trained. The Ministry has also made 
recommendations to modify the Personal Support Worker Training Program such that it would 
fund more basic training while still supporting the full certification program. 


SCREENING OF PERSONAL SUPPORT WORKERS 


Recommendation 

To assist in safeguarding the interests and well-being of long-term care community 
service recipients, the Ministry should ensure long-term care service agencies 
appropriately screen workers providing care. 


Current Status 

A working group comprising consumers, community agencies and ministry staff V 
recommended practices for the screening of personal support workers at the time of hiring and 
the ongoing screening of these workers. In the fall of 2000, provincial liaison meetings with the 
Ontario Community Support Association and the Ontario Association of Community Care 
Access Centres will include discussions on the implementation of the recommended practices. 
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Ministry of Health: 
Ontario Health Insurance Plan — 3.06 


BACKGROUND 


The Ontario Health Insurance Plan (OHIP), which was established under the Health 
Insurance Act, pays at specified rates for insured services provided to residents of Ontario by 
physicians and other health care providers, commercial laboratories, and diagnostic and 
therapeutic facilities. OHIP also pays for medical and hospital treatment provided to Ontario 
residents in other provinces and outside of Canada. 


Our 1998 audit assessed whether system controls and related procedures were adequate to 
ensure the proper approval, processing and payment of health care provider and commercial 
laboratory claims, and whether the Ministry had adequate policies and procedures in place to: 


* ensure that OHIP was managed with due regard for economy and efficiency and in 
accordance with applicable legislation; and 


* measure and report on the effectiveness of the OHIP system. 


CURRENT STATUS OF RECOMMENDATIONS 


Recommendations relating to the following areas of our 1998 report have been substantially 
implemented: 


¢ Claims Processing—Once-in-a-Lifetime Operations; 
¢ Out-of-Country Claims—Prior Approval, and Claims Verification; and 
¢ Investigation Unit. 


The current status of the remaining recommendations 1s as follows: 


REGISTERED PERSONS DATABASE 


Recommendation 


To better ensure that services are provided only to eligible individuals, the Ministry 
should complete the verification of persons registered prior to 1995. 


Current Status 


According to the Ministry, as of January 2000, there were approximately 12 million OHIP cards 
in circulation, including 3.7 million new cards issued under the more stringent rules implemented 
in 1994 and 1995. This includes about 270,000 residents re-registered under Primary Care 
Reform. This leaves approximately 8.3 million cards issued prior to 1995 for which eligibility is 
still to be verified. 


The Ministry advised us that as the remaining Primary Care Network sites confirm their 
agreements, their enrollees will also be asked to reconfirm their eligibility for health coverage. 
Province-wide re-registration remains timed to align with government corporate initiatives such 
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as the introduction of a government “smart card” and the development of an appropriate 
registration service delivery network. The timeframes are still to be determined. 


CLAIMS PROCESSING 
MANUAL CLAIMS CORRECTIONS 


Recommendation 


To help obtain assurance that appropriate and consistent corrective action is taken on 
rejected claims: 


* information on individual rejected claims that have been approved by the claims 
assessors Should be maintained for ready access by management; 


¢ management should regularly review the use of bypass codes; 


¢ any changes made in the On-line Claims Correction System (OCCS) to original claims 
data submitted by health care providers should be traceable; and 


¢ the identity of the claims assessors responsible for any changes made in the OCCS 
should be determinable. 


Current Status 


The Ministry now produces a report showing individual rejected claims and is planning to 
amend the report to include the identification of the claims assessor handling each claim. 


Staff training to help ensure assessment codes are used properly and consistently was 
completed in April 1999. Regarding managerial review of the use of bypass codes, the Ministry 
stated it produces a monthly report, which is available on-line to service managers. However, 
the Ministry advised us that, since the report is detailed and lengthy, it is a laborious and time- 
consuming task which is carried out periodically by management. The program management 
has asked that the creation of a more user-friendly report, which would streamline the review 
process, be identified as a systems development priority. The amended report will be required to 
be reviewed monthly. 


The Ministry stated that further analysis and systems changes are required to allow the tracing 
of changes to provider-submitted claims and that, since the Year 2000 system freeze has been 
lifted, these requirements will be addressed in its current claims project. 


Systems enhancements that permit the identification of the claims assessor who makes a wot 
change to original claims data or approves a rejected claim are scheduled for implementation in 


the year 2000. 


OUT-OF-COUNTRY CLAIMS 
STALE-DATED CLAIMS 


Recommendation 


The Ministry should implement system controls an 
are not paid unless they meet the Health Insurance , 
providers submit claims within six months of the date of service. 
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Current Status 


The Ministry wishes to reduce the timeframe for claims submission to within four months from 
the date of service rather than six months. Upon senior management approval, which is 
expected shortly, a revised list of extenuating circumstances and relevant procedures for stale- 
dated claims will be forwarded to processing offices. 


For out-of-country claims, the Ministry has changed its policy to allow claims submitted within 
12 months of the service delivery. 


Ministry staff indicated that legislation would be prepared to enact these changes. 


MONITORING AND CONTROL UNIT 
PHYSICIANS MONITORING SYSTEM 


Recommendation 


To better highlight questionable billing practices, the Ministry should ensure that 
screening tools are developed to replace the Physicians Monitoring System. 


Current Status 


Work is now underway to develop and implement claims analysis tools, such as a data-mining 
tool to screen, monitor and analyze provider OHIP claims. Implementation is expected within 
the next two years. As an interim solution, the Ministry has been using a temporary monitoring 
application developed by Queen’s University. 


VERIFICATION LETTERS SYSTEM 


Recommendation 


The Ministry should review the effectiveness of the Verification Letters System to ensure 
that it meets the objective of detecting abuse and deterring fraudulent claims. In order to 
enhance the possibility of obtaining better information for analysis purposes, the 
Ministry should also consider including in the verification letters a description of the 
services rendered in non-medical terms. 


Current Status 


The Ministry initiated a number of changes to increase the effectiveness of its Verification 
Letters System. This included doubling the number of letters sent to patients to verify that 
services had been provided and installing an automated tracking system to monitor the return of 
letters and a toll-free number to answer questions from letter recipients. 


An overall review of the Verification Letters System and the development of a strategic plan 
have been initiated. The review was completed in January 2000. An integrated replacement 
system will be prioritized for development and will include better descriptions and improved 
flexibility to modify text. 


282 Office of the Provincial Auditor 


LLL 
REFERRALS TO THE MRC (MEDICAL REVIEW COMMITTEE) 


Recommendation 


To deter misuse of the system by health care providers and to expedite the recovery of 
inappropriate billings, the Ministry should: ; 


* exercise its full authority under the Health Insurance Act with respect to sanctions and 


assess the need for further sanctions on those health care providers who are found to 
be repeatedly abusing the system; 


* hold orientation sessions and provide reference material to help reduce the incidence 
of incorrect billings resulting from the misinterpretation of service codes in the 


Schedule of Benefits; 


* assess the referral process and work with the Medical Review Committee to improve 
the timeliness of its reviews; 


¢ fill MRC vacancies to decrease the backlog of cases and to expedite the review of 
new referrals; and 


¢ request the MRC to reinstate the practice of publishing its recommendations, thus 
helping to prevent inappropriate treatments and billings. 


Current Status 
We were advised by the Ministry that actions taken since our audit included the following: 


¢ Anew regulation under the Health Insurance Act introduced financial sanctions where the 
MRC finds that claims were submitted inappropriately. This regulation included a one-time 
settlement opportunity for physicians that were before the MRC. It is anticipated that this 
may reduce the MRC backlog by as many as 100 cases. 


¢ Policy development is under way regarding the publication of MRC information as 
authorized under the Health Insurance Act. 


- A Physician Starter Kit was developed to train physicians in the use of the Schedule of 
Benefits for Physicians. In 1999, the Kit was introduced as a pilot to a selected group of 
graduating residents in family medicine. In the year 2000, the program will be expanded to 
all graduating residents and to all newly OHIP-registered physicians coming from out of the 
province. 


¢ MRC vacancies were filled. 


* A joint review of the MRC process by the Ministry and the College of Physicians and 7 
Surgeons of Ontario is being incorporated into a one-year project reviewing all monitoring 
and control processes. 


PARAMETERS OF PRACTICE 


Recommendation 


To improve patient care and help ensure that provincial funding for health care is utilized 
economically and effectively, the Ministry should facilitate the development of additional 
parameters of practice in the health care professions. 
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The Ministry should pursue sharing information with the College of Physicians and 
Surgeons of Ontario. The use of any information should be clearly defined and within 
legislated limitations. 


Current Status 


Quality assurance regulations have been passed for all 21 regulatory colleges. The Health 
Professions Regulatory Advisory Council is undertaking an assessment of the colleges’ quality 
assurance programs and will advise the Minister in a final report to be issued in the year 2000. 


Under the new four-year Ontario Medical Association (OMA)/ministry agreement, effective 
April 1, 2000, the joint OMA/Ministry Guideline Advisory Committee will continue to develop 
and recommend appropriate strategies for the implementation and monitoring of practice and 
referral guidelines and will continue to make recommendations for assisting in the 
implementation of prescribing guidelines. 


Ministry of Municipal Affairs and 
Housing and Ontario Housing 
Corporation: 

Rent Supplement Programs — 3.07 


BACKGROUND 


In 1997, the provincial and federal governments provided rent subsidies of over $131 million to 
permit eligible households to obtain affordable accommodation in the private sector and in 
certain federally supported non-profit and cooperative housing projects. Rent supplement units 
in private sector properties were administered by the Ontario Housing Corporation (OHC) and 
its 54 local housing authorities. Rent supplement units in the federally supported non-profit and 
cooperative housing projects were administered by the Ministry of Municipal Affairs and 
Housing. 


In 1998 we assessed whether the Ministry and OHC had satisfactory systems and procedures 
in place to manage rent supplement programs economically, efficiently and in compliance with 
requirements, including procedures to measure and report on program results. 


CURRENT STATUS OF RECOMMENDATIONS 


The OHC and the Ministry have made significant progress in addressing our recommendations. 
Recommendations in the following areas have been substantially implemented: 


¢ Program Accountability and Results Reporting; 


¢ Eligibility and Rent Determination—Programs Administered by Local Housing Authorities; 
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* Negotiations and Agreements with Landlords—Negotiations of Guaranteed Rent and 
Properties with High Subsidies; and 


¢ Compliance with Legal and Maintenance Requirements. 


For some of the above actions more time was required to achieve the desired improvements. 
For example, a number of policy and administrative improvements are contained in a new rent 
supplement administration manual that was released to field staff in July 2000. Also, new rent 
supplement agreements with landlords that improve the ability of local housing authorities to 
ensure that guaranteed rents do not exceed market rents are being implemented as the old 
agreements expire or when additional units are obtained. 


Our other recommendations and the status of the actions taken to address them is as follows: 


ELIGIBILITY AND RENT DETERMINATION 
PROGRAMS ADMINISTERED BY REGIONAL OFFICES 


Recommendation 


To better ensure that the non-profit and cooperative housing groups comply with their 
rent supplement agreements, the Ministry should: 


¢ ensure that the results of reviews conducted by Canada Mortgage and Housing 
Corporation and comments reported to such housing groups by external auditors are 
communicated to regional offices on a timely basis; 


¢ develop a risk-based strategy that incorporates the information not currently 
obtained from external auditors and Canada Mortgage and Housing Corporation to 
plan and conduct compliance reviews for all rent supplement programs, and 


* explore options for corrective actions when groups fail to comply with important 
requirements of their agreements. 


Current Status 

Under the new Social Housing Agreement signed with the federal government in November 
1999, all non-profit housing groups became the responsibility of the province to both fund and 
administer. Cooperative housing groups to which rent subsidies were paid by the Ministry are 
now funded and administered entirely by Canada Mortgage and Housing Corporation. The 
Ministry has taken recent steps to better coordinate administrative processes with those of 
Canada Mortgage and Housing Corporation and to facilitate the transfer of responsibilities. 


Options for corrective actions when groups fail to comply with important requirements of their 
agreements will be addressed as part of program reform currently underway and the social 


housing devolution legislation expected in 2000. 
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COORDINATION WITH THE MINISTRY OF COMMUNITY AND SOCIAL 
SERVICES 


Recommendation 


To better ensure that households are not being over-subsidized, the Ontario Housing 
Corporation should establish information-sharing arrangements with the Ministry of 
Community and Social Services. 


Local housing authorities should use the information to determine whether individuals 
who have been investigated and have had their social assistance terminated or reduced 
are also ineligible for rent subsidy or paying less rent than required. 


Current Status 


Progress in addressing this recommendation has been slow as an information-sharing 
agreement between the Ministry of Community and Social Services and Ministry of Municipal 
Affairs and Housing was not signed and submitted to the Information and Privacy 
Commissioner for review until February 2000. On March 29, 2000, the Information and Privacy 
Commissioner requested the Ministry to clarify certain aspects of the agreement. The Ministry 
has responded with clarification on the items noted and was awaiting formal confirmation from 
the Commissioner that the agreement complies with the Freedom of Information and 
Protection of Privacy Act. 


Ministry of Natural Resources: 
Financial Controls Review — 3.08 


BACKGROUND 


The mandate of the Ministry of Natural Resources is to achieve the sustainable development of 
the province’s natural resources, including the development of the economies and communities 
that depend on these resources. The Ministry’s goals are to ensure the long-term health of 
ecosystems by conserving soil, aquatic, forest and wildlife resources. It is also responsible for 
the protection of people and property from the threat of forest fires, floods and soil erosion. 


To accomplish these goals, the Ministry spent $521 million during the 1997/98 fiscal year, and 
collected $448 million in revenue. Expenditures consisted of $252 million for staff salaries and 
benefits and $269 million for other expenditures, which included primarily the purchase of 
supplies, services and equipment. 


In 1998 we assessed the adequacy of the Ministry’s financial controls, systems and procedures 
for ensuring that expenditures were properly authorized, processed and recorded and that 
revenues were properly billed, collected and recorded. 


We found that the Ministry’s financial controls, systems and procedures required significant 
strengthening to ensure that expenditures were properly authorized, processed and recorded. In 
addition, financial controls over water power fees required strengthening, but the controls over 
the billing, collection and recording of other revenues were adequate. 
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CURRENT STATUS OF RECOMMENDATIONS 
The Ministry substantially implemented recommendations relating to the following areas: 


* Accountable Advance Account—Account Reconciliations, Segregation of Duties, Cheque 
Signing Controls, and Employee Advances; 


¢ Payroll Expenditures; 


* Other Expenditures—General Expenditure Controls, Procurement Practices, and Travel 
Expenses and Purchase Cards; and 


* Revenue—General Revenue Controls, and Forestry Trust Funds. 


The current status of the remaining recommendation is as follows: 


REVENUE 
WATER POWER REVENUE 


Recommendation 
In order to ensure the receipt of all the water power fees it is entitled to, the Ministry 
should: 


¢ establish procedures to bill and collect in accordance with legislation and the legal 
agreements; and 


¢ periodically verify, at power company sites, that the information submitted is 
accurate. 


Current Status 


The Ministry has compiled a complete water power site list to ensure that all revenues are 
billed and collected. In addition, the Ministry is in the process of establishing a financial audit 
function to verify information submitted by the power companies. It plans to have the procedure 
in place in the fall of 2000. 


Ministry of Natural Resources: 
Fish and Wildlife Program — 3.09 


BACKGROUND 


The mandate of the Ministry of Natural Resources is to achieve the sustainable development of 
the province’s natural resources, including the development of the economies and communities 
that depend on these resources. The goal of the Fish and Wildlife Program 1s to maintain and, 
where possible, enhance the social, cultural, economic and environmental benefits derived from 
the province’s fish and wildlife resources. For the 1997/98 fiscal year, total funding for the Fish 


and Wildlife Program was $72 million. 
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Our 1998 audit assessed whether adequate systems, policies and procedures were in place to 
measure and report on the effectiveness of the Program and to identify areas where corrective 
actions were required; to ensure compliance with legislation and ministry policies; and to ensure 
that resources were managed with due regard for economy and efficiency. 


We found that the Ministry did not have adequate procedures in place to provide the 
information necessary for measuring and reporting on the Program’s effectiveness in sustaining 
fish and wildlife resources or to identify areas where corrective actions were required. The 
Ministry also needed to improve its resource management and enforcement practices to ensure 
compliance with legislation and ministry policies and to ensure that resources were managed 
economically and efficiently. 


Since our 1998 audit, the Game and Fish Act was replaced by the Fish and Wildlife 
Conservation Act, which became effective January 1, 1999. 


CURRENT STATUS OF RECOMMENDATIONS 


The Ministry has taken some action on all the recommendations we made in 1998 and has 
substantially implemented recommendations related to the following areas: 


¢ Wildlife Management—Wildlife Harvest Management; 
¢ Fisheries Management—Fish Stocking Methods; 


* Compliance with Legislative and Ministry Policies—Enforcement Activity, Compliance 
Activity and Violation Reporting System, and The Fish Inspection Act; and 


- Due Regard for Economy and Efficiency—Administrative Time, and Fish and Wildlife 
Special Purpose Account. 


With respect to our other recommendations, the status of action taken is as follows: 


MEASURING AND REPORTING ON EFFECTIVENESS 


Recommendation 


To ensure that the program is effective in meeting its stated objectives and to identify 
areas where corrective actions are required, the Ministry should develop sound 
performance measures that are linked to the overall objectives, perform the necessary 
assessments and periodically report on the programs achievement in sustaining fish and 
wildlife resources. 


Current Status 


The Ministry has identified key results for the program along with the desired outcomes and the 
associated measures. However, the Ministry is continuing to develop measurements and 
improve its information gathering to enable it to assess and report quantitatively on the 
sustainability of fish and wildlife resources and on the social and economic benefits derived 
from the use of these resources. For moose, deer, bear and fur bearing animals, the Ministry is 
currently investigating methods for timely reporting of actual harvests compared with 
sustainable harvest limits for wildlife management units. In the fisheries area, a number of 
activities have been put in place to enhance the Ministry’s ability to provide regular status 
reports on the sustainability of the resources. 
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WILDLIFE MANAGEMENT 


WILDLIFE MANAGEMENT POLICIES 


Recommendation 


To ensure that wildlife populations are maintained at sustainable levels, the Ministry 
should: : 


* develop and implement the necessary wildlife management policies; 
¢ update the desired population levels or status for each management unit; and 


¢ set harvest targets based on reliable and current animal population and status 
information. 


Current Status 


The Ministry has carried out some work for each of the three big game species (moose, deer 
and bear) in an attempt to maintain populations at sustainable levels. With respect to moose, the 
Ministry is conducting aerial population surveys on a more regular basis as part of a provincial 
plan to enhance the assessment of desired population levels. During the 2000/01 fiscal year, the 
Ministry will revise provincial population target levels based on a review of current population 
levels and consultations with the Ministry science and operational staff, the Fish and Wildlife 
Advisory Board, the Big Game Management Advisory Committee and the Ontario Moose Bear 
Allocation Advisory Committee. 


In 1998, the Ministry began a deer hunt review to identify areas where the deer population can 
support an increased harvest on a sustainable basis. This review was intended to facilitate the 
development of a deer management policy. Public consultations will be initiated during the 
2000/01 fiscal year to obtain additional input for a provincial deer policy. 


The Ministry feels that the current bear management policy is adequate to ensure that 
populations are maintained at sustainable levels. Improvements are being made to gather 
current bear population status and harvest information. For example, starting with the 2000 bear 
hunt, postcard surveys will be included with the resident hunting licence to improve survey 
coverage. In addition, the Ministry is currently reviewing the provincial bear program, which 
may identify a need for further development of policies and procedures. 


WILDLIFE POPULATION ASSESSMENTS 


Recommendation 
To properly manage wildlife and help ensure that the sustainability goal is achieved, the 
Ministry needs to: 


* carry out population assessments more frequently to accurately determine the 
populations of the various wildlife species; 
* analyze the survey information and report the results on a more timely basis; and 


- require district offices to use the results for decision-making purposes and the effective 
management of area wildlife resources. 


289 
Special Report: Accountability and Value for Money 





Current Status 


The Ministry has substantially implemented the recommendation regarding more frequent 
assessments of moose and deer populations. For bear, the Ministry has maintained the annual 
assessment using the provincial bear population index network and has improved harvest 
monitoring. The Ministry has committed to improving the timing of harvest surveys and 
analysis. 


In an effort to help process population assessments, perform more timely analysis and set 
harvest quotas, the Ministry continues to develop a Big Game Management Information 
System. Along with population target workshops for staff, the Ministry’s intent is that this 
system will help generate reliable information to manage big game species on a sustainable 
basis. 


FISHERIES MANAGEMENT 
RECREATIONAL FISHERIES MANAGEMENT 


Recommendation 


In order to make better-informed decisions regarding the management of fish 
populations, the Ministry should undertake cyclical assessments of a representative 
sample of provincial lakes. 


Current Status 


The Ministry has initiated a broad scale review of inventory, monitoring and assessment 
activities for all programs including recreational fisheries. In addition, the Ministry is in the 
development stage of designing a monitoring program for provincially significant fish species, 
such as lake trout and walleye. This program is to include a random selection of a 
representative sample of provincial lakes that will be monitored to evaluate the health of fish 
populations. 


FISH STOCKING PROGRAM 


Recommendation 


To help ensure that the fish stocking program is rehabilitating natural populations and 
encouraging economic spin-offs, the Ministry should perform regular assessments to 
determine whether the lakes and species currently being stocked meet the objectives for 
rehabilitation, introduction and put-grow-take stocking. 


Also, to help protect the natural fish stocks and to continue to provide a sustainable 
population, the Ministry should revise its fish stocking policy to reflect current scientific 
research. 


Current Status 


The Ministry carries out sampling programs on the Great Lakes to evaluate whether fish 
rehabilitation objectives are being met. However, assessment of rehabilitation and put-grow- 
take stocking is not fully implemented on inland lakes. Beginning in the 2001/02 fiscal year, the 
Ministry plans to initiate efforts to develop provincial standards for the assessment of stocked 
fish in inland lakes. 
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In addition, for the fish stocking policy, the Ministry has developed interim guidelines for use at 
the local level and is currently developing more comprehensive guidelines for fish stocking in 


Ontario. Guidelines will be based on the best science currently available and will be 
implemented in the 2001/02 fiscal year. 


COMPLIANCE WITH LEGISLATION AND MINISTRY POLICIES 
HUNTING AND FISHING LICENCE SUSPENSIONS 


Recommendation 


In order to properly track suspended individuals and to make the suspension system more 
effective, the Ministry should: 


¢ enhance the system to help conservation officers on patrol identify suspended 
individuals; 


¢ ensure that conservation officers input all the required information into the 
Compliance Activity and Violation Reporting System; and 


¢ implement procedures to prevent suspended individuals from obtaining a licence. 


Current Status 


While the Ministry has enhanced the system, new methods and technologies are being explored 
to put “live” licence suspension information into the hands of officers on patrol. In addition, the 
processes and policies for the handling of licences that have been suspended are currently 
being reviewed and rewritten. 


With regard to ensuring that conservation officers input all the required information into the 
Compliance Activity and Violation Reporting System, the Ministry has linked the timely input of 
data to employee performance targets and implemented a tracking system to monitor 
performance and has a draft policy in the approval stages that would require data to be input 
within a specific timeframe. 


To prevent suspended individuals from obtaining a licence, the Ministry is investigating 
appropriate procedures to allow Ministry Outdoor Card Information System (OCIS) licence 
issuers to check the suspension status of individuals seeking to purchase a licence. The Ministry 
remains open to exploring methods that could extend suspension status checks to outside 
licence issuers while remaining in compliance with the Freedom of Information and 
Protection of Privacy Act. 
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Ministry of Natural Resources: 
Science and Information Resources 
Division — 3.10 


BACKGROUND 


The mandate of the Ministry of Natural Resources is to achieve the sustainable development of 
the province’s natural resources, including the development of the economies and communities 
that depend on these resources. The Science and Information Resources Division of the 
Ministry provides leadership in the development and application of scientific knowledge, 
information management and information technology, primarily in the two program areas of fish 
and wildlife, and forest management. For the 1997/98 fiscal year, the Division employed 
approximately 500 staff, and its expenditures totaled $63.5 million. 


Our 1998 audit assessed whether program resources were properly managed with due regard 
for economy and efficiency and whether satisfactory procedures were in place to measure and 
report on the effectiveness of the Division’s activities. 


We found that the Science and Information Resources Division did not ensure that certain 
program resources were adequately managed with due regard for economy and efficiency, and 
satisfactory procedures were not in place to measure and report on the effectiveness of the 
Division’s activities. Specifically, the Ministry needed to implement standardized scientific 
research and development life cycle processes. With regard to information resources and 
technology, we found that the Ministry’s administrative procedures required significant 
improvement to ensure compliance with mandatory government policies and the Ministry’s own 
procedures. As a result, we made a number of recommendations for improvement. 


CURRENT STATUS OF RECOMMENDATIONS 


The Ministry has taken some action on all of the recommendations we made in 1998 and has 
fully or substantially implemented recommendations related to the following areas: 


¢ Information Systems Management—Identification of Information Needs, Project Planning 
and Selection, and Consulting Services; and 


¢ Information Technology Management—Computer Needs Analysis, Lease Agreements, 
Management Board Secretariat Approval, Management of Information Technology Leases, 
and Management of Information Technology Assets. 


With respect to our other recommendations, the status of action taken is as follows: 
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SCIENCE DEVELOPMENT ACTIVITIES 
SETTING DIRECTION AND SCIENCE PRIORITIES 


Recommendation 


To ensure that scientific research contributes to the effective management and 
sustainable development of the province s natural resources, the Ministry should: 


¢ implement the processes outlined in the October 1996 Strategic Plan for Science and 
Technology; 


¢ develop clear research priorities in consultation with the program areas; and 


¢ establish clear relationships with program areas that hold the Science Development 
and Transfer Branch accountable for the delivery of research results that meet their 
users’ needs. 


Current Status 

The Ministry is in the process of implementing a Forest Science Strategy that focuses on 
matching the needs of the Ministry’s Forest Management Program to its forest science and 
technology activities. The objective is to serve sustainable forest management goals and 
legislative responsibilities. 


Many of the processes outlined in the Branch’s 1996 strategy plan are being addressed through 
the implementation of the Forest Science Strategy, including the establishment of science 
priorities, management of human and financial resources, quality assurance and reporting. 
Research priorities will be established in consultation with program areas, and performance 
measurement tools are to be put in place. This process is being implemented during the 2000/01 
fiscal year for the Forest Management Program. Subsequently, the process will be implemented 
for the Fish and Wildlife Program. 


PROJECT SELECTION 


Recommendation 

To ensure the selection of those science projects that best achieve the Ministry's objective 
of the sustainable development of the province s natural resources, the Science 
Development and Transfer Branch should: 


¢ implement standard project proposal requirements which include program area input, 
the expected time to complete the project with critical interim milestones, the estimates 
of the full cost of the project, the anticipated results and the likelihood of success; 


- develop clear criteria for the selection and approval of projects for funding; 


¢ annually evaluate each project to determine if funding should be continued, modified 


or terminated; and 


¢ document the rationale for selecting new proposals and the decisions regarding 


ongoing projects. 
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Current Status 


The Ministry is implementing science working groups that are developing strategic priorities. In 
making decisions, science staff will have to demonstrate how project proposals relate to these 
strategic priorities. To help evaluate and prioritize projects, the Ministry is implementing a 
decision process that requires the use of a standard project proposal template. The template will 
be tied into the annual work planning cycle and include a statement of clear criteria for project 
selection, as well as documentation for project evaluation and decision making. In addition, the 
Ministry has developed a system to track in-year and multi-year project milestones and funding 
commitments. 


The new decision process to evaluate projects will be implemented in the 2000/01 fiscal year 
for forest science projects. In the 2001/02 fiscal year, fish and wildlife projects will also be 
subject to the new decision process. 


RESEARCH MONITORING AND REPORTING 


Recommendation 


To ensure that all projects are progressing as expected and continue to be relevant to the 
program needs, the Ministry should: 


¢ develop standardized monitoring procedures which include the tracking of critical 
reassessment milestones; 


¢ ensure that the annual science project summaries contain sufficient detail to assess 
progress to date and the likelihood of achieving the expected results; and 


* implement post-project evaluation procedures to determine if completed science 
projects benefited the program areas. 


Current Status 


The Ministry’s implementation of the decision process described under the Project Selection 
recommendation 1s intended to address these concerns. These procedures are to include in-year 
financial tracking of costs, while the annual funding reviews will track and report results of 
projects over their life cycles. In addition, the Ministry intends to evaluate the impact of 
completed science projects on the program areas and the Ministry. 


RESEARCH FUNDING 


Recommendation 


To ensure a balance between annual or short-term funding and the requirements of long- 
term research projects, the Ministry should consider a funding model that includes the 
full life cycle of projects and addresses fluctuations in funding requirements. 


Current Status 


The Ministry intends to be tracking funding over the projects’ life cycles using its new decision 
process model, which will select, monitor and evaluate science projects. The model will take 
into account both in-year and multi-year funding requirements. 
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INFORMATION SYSTEMS MANAGEMENT 
PROJECT MONITORING 


Recommendation 


To ensure the successful completion of information technology projects on time and on 
budget, the Ministry should develop a formal monitoring process to track project costs 
and require status reports that include progress toward the milestones, related 
deliverables and benefits stated in the project plan. 


Current Status 


The Ministry has developed new policies, standards, processes and documentation requirements 
for managing projects. As part of this initiative, the process and standards for monitoring and 
reporting on projects costs and milestones have been developed. The Ministry intends to start 
using the formal monitoring process during the 2000/01 fiscal year. 


Ministry of the Solicitor Generali ana 
Correctional Services: 
Office of the Fire Marshal — 3.11 


BACKGROUND 


The primary function of the Office of the Fire Marshal (OFM) is to minimize the loss of life and 
property from fire by helping municipalities and fire departments improve their fire protection 
and prevention services. The Fire Protection and Prevention Act, which came into force in 
October 1997, provides the Fire Marshal with the authority to monitor, review and advise 
municipalities respecting their provision of fire protection services and to make 
recommendations to municipal councils for improving the efficiency and effectiveness of those 
services. For the 1997/98 fiscal year, program expenditures were $23 million, of which 70% 
was for staffing. The OFM had approximately 220 staff as of March 31, 1998. 


On the basis of our audit, we concluded that current effectiveness measures were not 
sufficiently comprehensive to assess the effectiveness of municipal fire services or OFM 
programs. 

We also concluded that, on an overall basis, the OFM had satisfactory systems and procedures 
in place to promote compliance with fire safety legislation and efficient and effective municipal 
fire services. However, improvements were required to deal with violations of the Ontario Fire 
Code and Hotel Fire Safety Act that were identified in over 40% of fire safety inspections. 


Since June 1998, the program has been administered by the Ministry of the Solicitor General. 
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CURRENT STATUS OF RECOMMENDATIONS 


Although the OFM has taken actions to implement our recommendations, these actions were 
still in progress. Action taken and planned for by the OFM based on our 1998 recommendations 
is as follows: 


MEASURING AND REPORTING ON PROGRAM 
EFFECT VENE Se 


Recommendation 


In order to generate accurate and appropriate information for decision making and to 
minimize the loss of life and property from fire, the Ministry should improve the 
measurement and reporting of the effectiveness of the Office of the Fire Marshals and 
municipalities’ programs and efforts. 


In addition, the Ministry should develop effectiveness measures for the oversight of 
municipal fire services recently mandated to the Fire Marshal by the Fire Protection and 
Prevention Act. 


Current Status 


In September 1999, the Ontario Fire Marshal initiated a long-term program called the Shaping 
Fire Safe Communities Program (SFSCP). The SFSCP replaces previous programs that were 
in development and consists of two major components: 


¢ A comprehensive survey will be used to determine the extent and scope of the fire 
protection delivery system in each municipality. Implementation of the survey is expected to 
begin during the fall of 2000. The information collected will form the basis of a 
comprehensive database. 


¢ A how-to kit, expected to be released in May 2001, will give municipalities and fire 
departments the tools they need for making informed choices about fire protection based on 
local needs and circumstances and for achieving the minimum mandatory requirements 
under the Fire Prevention and Protection Act. 


Using information gathered from the SFSCP, the OFM intends to develop benchmarks and 
performance measures for use by municipalities and fire departments. The OFM will also be 
developing performance measures for their oversight role. 


COMPLIANCE WITH FIRE SAFETY LEGISLATION 
PROPERTY OWNER COMPLIANCE WITH FIRE SAFETY LEGISLATION 


Recommendation 


To improve compliance with fire safety legislation by property owners, the Office of the 
Fire Marshal should examine, in conjunction with other stakeholders, existing and new 
options for enforcement of the Ontario Fire Code and for education of property owners 
regarding their responsibility for fire safety. 
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Current Status 


To improve the quality of inspections conducted by the OFM, supervisors are now required to 
conduct random audits of properties to assess inspectors’ performances. Any deficiencies 
noted require the supervisor to institute corrective measures. As well, since July 1998, ticketing 
has been in place under the Provincial Offences Act for smoke alarm violations. In addition, in 
June 2000, the OFM issued guidelines to the municipal fire services that focus on the 
mandatory provisions of the Act, including requirements for inspection and enforcement. 


The OFM has developed a number of educational and training packages for facility owners, 
prosecutors and the municipal fire services that are aimed at raising the level of compliance. In 
addition, as part of the how-to kit under the SFSCP, the fire service is expected to identify 
strategies to include community partnerships and sponsorships in fire safety programs. 


INFORMATION ON INSPECTION ACTIVITIES 


Recommendation 

In order to generate accurate and appropriate information for decision making to 
minimize the loss of life and property from fire, the Office of the Fire Marshal should 
improve the quality of its information on fire safety inspections. It should then use the 
information obtained to help ensure that: 


* higher risk properties are maintained at an acceptable fire safety standard; 
¢ common fire safety deficiencies are identified to better focus preventive efforts; and 


° best practices are identified and shared with other municipalities. 


Current Status 

For its inspection program the OFM has acquired new software, which will be utilized to 
identify common violations and compliance rates on first inspections and re-inspections. This 
will allow the supervisors to develop strategies to improve compliance. The OFM plans to share 
the identification and correction process for common violations with municipal fire services. 


The survey that is being developed as part of the SFSCP will assist in gathering information on 
risk assessment strategies, inspection targeting and public fire safety education needs. The 
how-to kit will include features that support the sharing of best practices among fire 
departments. 


INSPECTIONS CONDUCTED BY THE OFM 


Recommendation 

In order to eliminate inefficiencies and to ensure that responsibilities under the Fire 
Protection and Prevention Act are clearly delineated, the Office of the Fire Marshal 
should expedite the transfer of responsibility for its remaining inspection activities to 


municipalities. 


Current Status 
As part of the OFM’s ministry business planning activities, direction was sought and received to 
begin the mandatory transfer to municipalities of all hotels it currently inspects commencing 
fiscal 2000/01. A plan has been developed to complete this transfer within 18 months of 
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implementation. To ensure that a high standard is maintained for hotel inspections, the OFM has 
provided training to municipal fire services. The OFM’s municipal fire service monitoring 
process will be used to ensure that hotel inspections are continued after the transfer. 


Where a high level of inspection skills and abilities exists within a municipality, other facilities 
will be transferred, including homes for the aged and nursing homes. The OFM has initiated 
assessments of municipalities’ fire services that have accepted responsibility for inspecting 
nursing homes. Results so far have been favourable. 


MONITORING OF MUNICIPAL FIRE SERVICES 
MASTER FIRE PLANS 


Recommendation 


The Office of the Fire Marshal should more actively encourage and promote the need for 
municipalities, in conjunction with their fire departments, to periodically prepare master 
fire plans and should review those plans to help ensure that they provide timely and 
relevant information for assessing the adequacy of municipal fire services and fire 
prevention activities. These reviews could also be used to help determine the existence of 
any serious threats to public safety as well as situations requiring corrective actions. 


Current Status 


Under the SFSCP, if the risk assessment and evaluation indicates that there are significant gaps 
in the fire protection delivery system, the Fire Protection Advisors will work closely with the 
municipality to develop an action master plan to bring the delivery system up to an appropriate 
level. 


Three master fire-planning courses for smaller communities have been delivered, and two other 
courses to be delivered in the field are already planned for 2000/01. In addition, the OFM issued 
a guideline on master fire planning on March 31, 2000. The proposed monitoring survey and 
scanning by staff will identify the number of communities involved in master fire planning. 


ANNUAL SURVEY 


Recommendation 


The Office of the Fire Marshal should remind municipalities and their fire departments of 
their duty under the Fire Protection and Prevention Act to provide timely information on 

the fire protection and prevention services in their communities, or, alternatively, the 
Ministry should consider a regulation to more specifically mandate responses to the 
Office of the Fire Marshal’s annual survey. 


Current Status 


No further surveys have been conducted since our 1998 audit. The surveys about to be 
conducted under the SFSCP to gather core data on the fire service and the monitoring project 
will have more direct involvement by field staff. This will not only increase the response rate 
but also provide more accurate and complete information. The legislated requirements for 
municipalities under the Act, including reporting requirements, will form part of the how-to kit. 
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FIRE DEPARTMENT REPORTING PRACTICES 


Recommendation 


To facilitate the monitoring of municipal fire services by the Office of the Fire Marshal, 
municipal councils and fire chiefs, the Fire Marshal should take measures to assist local 
fire departments to improve their performance measuring, benchmarking capabilities and 
reporting practices. 


Current Status 


As part of the SFSCP, benchmarking, performance measures and best practices will be 
introduced to the fire service. Training at the Ontario Fire College for chief officers also 
includes a session on performance measures. 


The OFM field staff have also been trained on the concepts of performance measurement and 
benchmarking. Staff are now involved in benchmarking initiatives by the Ministry of Municipal 
Affairs and Housing and the municipal regional chief administrative officers. In addition, the 
OFM is conducting research on the performance measurement and reporting practices of other 
organizations. 


IDENTIFYING FIRE SAFETY CONCERNS 


FIRE INVESTIGATIONS 

An internal review conducted by the OFM in October 1997 determined that several 
investigators had significant backlogs of uncompleted investigations. Efforts were underway 
during our 1998 audit to eliminate the backlog and introduce new reporting procedures to better 
monitor progress on investigations. 


Current Status 


New reporting procedures were put in place to better monitor progress on investigations. 
However, the OFM has not made adequate progress to reduce its backlog of uncompleted 
investigations. As of August 2000, over 550 investigations were overdue for updating by 
investigators, and final reports were overdue for over 150 investigations. 


As a result of our follow-up, senior management has implemented additional controls and 
reporting requirements to ensure that the number of uncompleted investigations are reduced 
over the next few months and then maintained at more appropriate levels. 


FIRE LOSS REPORTING 


Recommendation 

To ensure that fire loss reporting is timely, relevant and efficient, the Office of the Fire 

Marshal should: 

* work toward having only one comprehensive database for fire loss reporting by fire 
departments and Office of the Fire Marshal fire investigations; 

* introduce measures to ensure that fire departments submit fire loss reports in a time 
manner, preferably using an electronic reporting format; 
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* ensure that fire departments and Office of the Fire Marshal investigators report the 
extent to which intoxication by alcohol and drugs was a circumstance contributing to 
fires. 


The Office of the Fire Marshal should also consider whether awareness programs to 
address intoxication as a fire safety issue are warranted. 


Current Status 


The necessary hardware and software programs have been purchased to allow for integrating 
the two current databases. Initially, municipal fire loss data will be converted to the new 
system. This work is being planned for the 2000/01 fiscal year. 


A new program that checks electronic fire loss data received from municipal fire departments 
has been developed and is currently in use. New methods of filing electronic data, such as via 
the Web, are also being researched. 


The fire service training programs now focus more on reaching and teaching high-risk and 
hard-to-reach groups in the community, such as those individuals intoxicated by drugs and 
alcohol. In addition, public service announcements and public educational programs now carry 
messages regarding drinking and smoking and/or drinking and cooking. The educational 
messages usually focus on those who live with the drinkers. 


Ministry of the Solicitor General and 
Correctional Services: 
Ontario Provincial Police — 3.12 


BACKGROUND 


Under the Police Services Act, the Ontario Provincial Police is responsible for policing areas of 
Ontario that do not have their own police agencies. The OPP is also responsible for patrolling 
traffic on certain highways, maintaining specialized investigative and enforcement capabilities to 
assist municipal police agencies, and enforcing liquor and other laws as the Solicitor General 
may direct in serving provincial interests. 


In 1998 we assessed whether the Ministry had adequate procedures and systems in place with 
regard to the OPP to: 


* measure and report on the effectiveness of mandated community-oriented policing and 
traffic management services in the promotion of public safety; and 


* ensure that policing services were delivered with due regard for economy and efficiency. 
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CURRENT STATUS OF RECOMMENDATIONS 


The Ministry has substantially implemented recommendations relating to the following areas: 


¢ Effectiveness Measures—Traffic Management—Effectiveness Assessment, and Traffic 
Management—Detachment Traffic Management Plans; and 


* Human Resources Management—Report Writing—(Exploring New Technologies for 
Officer Data Entry). 


With respect to our other recommendations, the status of the actions taken is as follows: 


EFFECTIVENESS MEASURES 
COMMUNITY POLICING 


Recommendation 


In order to be more effective in serving the community through community policing 
activities, the Ontario Provincial Police should: 


¢ fully implement the process developed for identifying and prioritizing policing 
services to meet community service expectations; 


¢ identify and disseminate best practices in community policing among detachments, 
and 


* measure the effectiveness of community policing activities against established 
criteria. 


The Ontario Provincial Police should also periodically evaluate the progress of 
community policing implementation, taking corrective action where necessary to ensure 
that implemented initiatives are effective in helping communities achieve the objectives of 
reducing crime and victimization. 


Current Status 

The OPP has developed a Service Delivery Process and an evaluation tool kit to aid in 
identifying and establishing effective community policing activities. A manual that provides 
guidance on how to customize policing has been developed and provided to all detachments for 
identifying and prioritizing policing services to meet individual community needs and 
expectations. 


Best practices in community policing are now being gathered and shared among detachments 
through a resource centre library. This will be further enhanced through the internal search 
capabilities of a newly developed computer application called CPNet, which is expected to be 


operational in 2001. 


Community satisfaction surveys called “Policing For Results” have been conducted in local 
detachments and responses are being evaluated. The completed evaluation will be used to 


establish a baseline against which the OPP can measure its improvement efforts in the future. 


DE 
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HUMAN RESOURCES MANAGEMENT 
STAFF DEPLOYMENT 


Recommendation 


The Ontario Provincial Police should review current staff scheduling practices and revise 
them as necessary to ensure that officer hours worked are efficiently matched to the 
service requirements of the communities involved. 


Current Status 


The OPP is finalizing an agreement with the Ontario Provincial Police Association for the 
implementation of a shift-scheduling manual. The manual was developed jointly with the union 
to ensure officer hours are efficiently matched to service requirements. 


OVERTIME MANAGEMENT 


Recommendation 


To promote the appropriate use of overtime, the Ontario Provincial Police should 
establish better management controls to ensure that overtime hours are: 


* worked only on the basis of clearly identified and justifiable need; and 


* monitored so that, if necessary, appropriate corrective action can be taken. 


Current Status 


According to the OPP, review of overtime claim statements is now the responsibility of each 
region. The head office monitors overtime expenditure through its budget reviews. 


The OPP informed us that it has recently introduced a salary management system that will be 
able to better project overtime expenditures by tracking payments and accrued liabilities. In 
addition, it has newly created a Finance Committee to monitor and review overtime usage and 
to ensure that overtime is managed responsibly. 


DIFFERENTIAL RESPONSE UNITS 


Recommendation 


In order to realize the potential savings from the differential response units (DRU), the 
Ontario Provincial Police should determine and implement the mechanisms necessary to 
ensure that the differential response unit program is fully utilized. 


Current Status 


All the regions within the OPP reviewed their respective DRU programs with staff involved in 
the differential response units. The differential response units are used by the OPP to identify 
those calls for service for which dispatching an officer to the scene would not provide any 
additional benefit and to address those calls over the telephone. The DRU programs enable the 
regions to provide a more efficient deployment of police officers. However, the review 
indicated that the OPP was not able to fully utilize the DRU in certain areas due to community 
resistance. 
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PROVINCIAL REVENUES FROM MUNICIPAL POLICING 
SERVICES 


Recommendation 


To ensure that the costs of providing its services to municipalities are appropriately 
recovered, the Ontario Provincial Police should: 


* ensure that its system is adjusted to collect more detailed information to identify costs 
for recovery; and 


¢ work with the Ministry to establish adequate systems and procedures to more 
effectively bill and collect costs associated with municipal policing activities. 


Current Status 


A modified daily activity report form was implemented in January 1999 to enable the OPP to 
capture and track more detailed information that reflects the actual activities of its officers. In 
particular, this form reports actual hours spent by officers on individual activities so that the 
costs involved in delivering municipal or provincial policing activities can be separately identified 
for costing purposes. 


For more timely costing and billing to municipalities, a Web-based electronic version of the form 
is currently being piloted and implementation is expected in the later part of 2000. 


ee 
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